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Preface 



For the thirty-second year, the Research and Theory Division of the Association for Educational 
Communications and Technology (AECT) is sponsoring the publication of these Proceedings . This is 
Volume #2 of the 3 3'“ Annual Proceedings of Selected Papers On the Practice of Educational 
Communications and Technology’. This volume includes papers presented at the national convention of the 
Association for Educational Communications and Technology held in Anaheim, CA. Copies are available 
online at AECT.ORG. Volumes 1 and 2 are also available through the Educational Resources 
Clearinghouse (ERIC) system. 

This volume contains papers primarily dealing with instruction and training issues. Papers dealing with 
research and development are contained in the companion volume (Volume #1). 

REFEREEING PROCESS: Papers selected for presentation at the AECT Convention and included in these 
Proceedings were subjected to a reviewing process. All references to authorship were removed from 
proposals before they were submitted to referees for review. Approximately sixty percent of the 
manuscripts submitted for consideration were selected for presentation at the convention and for 
publication in these Proceedings . The papers contained in this document represent some of the most current 
thinking in educational communications and technology. 

Michael R. Simonson 
Editor 
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What are the factors that contribute to ineffective and limited use of 
Learning Management Systems (LMS) in the schools? 

Sinem Aslan 
Charles M. Reigeluth 

Indiana University 

Abstract 

Data from a previous study 1 show that, although the interviewed teachers were considered to be the ones who used a 
Learning Management System (LMS) most in their schools, their use of features and functions of the LMS was very 
limited and ineffective. More importantly, they used the LMS to enhance their traditional way of teaching, which 
was teacher-centered and aligned with industrial-age practices. This is a follow-up study examining the factors that 
contribute to the teachers’ ineffective and limited use of features and functions of LMSs. Based on the data from the 
previous study 1 , a secondary data analysis of the semi-structured interviews with seven teachers and one technology 
coordinator was conducted to reveal the factors that contribute to ineffective and limited use of the LMS by the 
teachers. 



Introduction 

Instructional technology has become a part of our educational lives (Cuban, 1986) and an interesting area in 
which to conduct research since the introduction of instructional radio in the 1920s and instructional television in the 
1950s. Since then, the availability of technology has increased in public schools remarkably. In 1981, there was one 
computer for every 125 students in the schools. In 1991, the availability increased to one for every 18 students, and 
in 2000, there was one for every five students in the schools (Christensen, Horn, & Johnson, 2008). 

Classrooms opened their doors to the technology as early as the 1950s; however, technology is still trying 
to enter the classrooms because there are so many constraints between the doors of the classrooms and the 
educational technology. In this paper, we will focus on the Learning Management System (LMS), which is 
considered one of the most recent promising educational technologies used in the schools (Gilhooly, 2001). We will 
present the factors that contribute to ineffective and limited use of LMS among interviewed classroom teachers and 
the technology coordinator of the school district. 

The definition of LMS might differ with regards to how it is being used or the perception of how it should 
be used. In a nutshell, we define LMS as an integrated educational software interface that has a number of essential 
educational components related to learning and teaching. Reigeluth, Watson, Watson, Dutta, Chen & Powell (2008) 
propose four major functions of information-age educational computing. These functions are (1) record keeping for 
student learning, (2) planning for student learning, (3) instruction for student learning, and (4) assessment for student 
learning. They also identify secondary functions of an information-age educational system as (1) communication, 

(2) general student data, (3) school personnel information and (4) LMS administration. 

Today, most LMSs have integrated components that can facilitate some of the information-age functions 
mentioned above. They have many outstanding features and functions that can help teachers to facilitate and 
improve students’ learning. However, availability of built-in information-age functions in LMSs does not mean that 
teachers are using all of those functions and features effectively and adequately. 

In fact, research shows that teachers are using educational technology to enhance teacher-centered 
instruction. Russell, Bebell, O’Dwyer and O’Connor (2003) point out that computer use is usually perceived as “a 
special event or an add-on to the traditional curriculum” (p. 3). Moreover, Pea (2000), Christensen et al. (2008), and 
Brown and Green (2008) report that despite the high investments on educational technology, their use in the schools 
is very limited and the effectiveness of the current level of integration is an important concern. 



1 Aslan, S., Huh, Y., Lee, D., & Reigeluth, C. M. (2010). The Role of Personalized Integrated Educational Systems in the Information-Age 
Paradigm of Education. Paper submitted to the journal: Education Research International. 
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Theoretical Framework 



Hew and Brush (2007) analyzed 48 previous studies that presented empirical research findings related to 
barriers to educational computing in K-12 schools. Based on these studies, the authors identified 123 different 
barriers to technology integration. They classified these barriers into 6 categories: “resources”, “knowledge and 
skills”, “institution”, “attitudes and beliefs”, “assessment” and “subject culture”. The authors provided a relative 
frequency graph of these categories as shown in Figure 1. 




Figure 1. Relative frequency in which the barriers were mentioned in the past studies, from Hew and Brush (2007). 

Figure 1 illustrates that “resources” is the first ranking barrier with a rank of 40%. The second ranking 
barrier is “knowledge and skills” with a rank of 23%. The percentages of the other categories are pretty low in 
relation to these two categories. Therefore, in this study, we will focus on these two top ranking barriers to the use of 
technology in K-12 schools. 

According to Hew and Brush (2007), without enough resources provided to the teachers, there is less 
opportunity for them to integrate technology into the curriculum. The researchers identified that “resources” have 
four different sub-categories, including “technology”, “access to available technology”, “time”, and “technical 
support”. Furthermore, teachers’ lack of “knowledge and skills” is an essential concern, since teachers need to have 
enough technology background in order to effectively use educational technology in the classrooms. 

Methods 

Data from a previous study 2 show that although the interviewed teachers were the ones who used the LMS 
most in their schools, their use of LMS features and functions was very limited and ineffective. More importantly, 
they used the LMS to enhance their traditional way of teaching, which had a teacher-centered focus. 

This is a follow up study to examine the factors that contributed to the teachers’ ineffective and limited use 
of the features and functions of the LMS. Participants in this study were seven classroom teachers from different 
subject areas and different grade levels and one technology coordinator in the school district. All these teachers were 
from the same school district but from three different high schools. Two of the schools had a traditional curriculum; 
however, one of them had a curriculum that was highly focused on technology-based activities and project-based 
learning, which are aligned with the information-age paradigm of education. Use of Moodle among the classroom 
teachers was investigated in this study since Moodle was the predominant LMS used in the school district. 

Existing data 2 from semi-structured interviews with the seven teachers and the technology coordinator were 
used to identify the factors that contributed to the interviewed teachers’ ineffective and limited use of Moodle. Data 



2 Aslan, S., Huh, Y., Lee, D., & Reigeluth, C. M. (2010). The Role of Personalized Integrated Educational Systems in the Information-Age 
Paradigm of Education. Paper submitted to the journal: Education Research International. 
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were gathered from seven classroom teachers and one technology coordinator using the semi-structured interviews. 
Each interview took about an hour to an hour and half for each of the participants. These interviews were audio- 
recorded and transcribed verbatim. 

The primary data analysis method was secondary content analysis. A coding sheet was created using the 
data from the semi-structured interviews with the technology coordinator and the seven classroom teachers in order 
to identify the factors that contributed to ineffective and limited use of Moodle. 

Results 

The findings illustrate that “resources” and “knowledge and skills” were the most significant factors that 
contributed to the limited and ineffective use of the LMS among these teachers, which is consistent with the 
theoretical framework by Hew and Brush (2007). 

With respect to the “resources” factor, although interviewed teachers had access to the technology 
(Moodle), the data show that three out of the seven teachers and the technology coordinator described a number of 
different technical problems they faced while using Moodle, which implies their need for technical support. In 
addition, the technology coordinator and five of the seven teachers pointed out that they would like to see some 
more integrated functions in Moodle, such as a real-time group collaboration tool (e.g. Web 2.0 technologies), a 
flexible rubric tool, a teacher collaboration tool, and a video conferencing tool. In addition, one of the teachers 
expressed teachers’ concerns about time to create instructional materials using Moodle. He described some of his 
colleagues’ hesitancy to use Moodle due to their limited time. 

With regards to the “knowledge and skills” factor, the interview data illustrate that the technology 
coordinator and five out of seven teachers expressed the need for training to learn the features and functions of 
Moodle. Since most of the teachers were not familiar with all the features and functions that Moodle provides, they 
used the LMS in a limited manner. 

The interview data also show that the teachers’ use of Moodle varied based on the type of the school. As 
mentioned in the methods section, only one of the schools had a technology- focused and project-based curriculum, 
which is aligned with the information-age paradigm of education. Two out of the seven teachers were from this 
school. Our analysis of interview data illustrates that these two teachers used Moodle with respect to the 
information-age functions identified by Reigeluth et al. (2008), and their activities involved a student-centered 
learning environment. Data from these two teachers show that they did not have any major problems with 
technology skills; however, they wanted to see more features and functions in Moodle, which implies their need for 
a more integrated and functional educational technology. 

Discussion and Future Research 

Another important question emerged from the interview data. There are schools that have undergone 
paradigm change from the industrial-age to the information-age in the United States. In these schools, technology 
use is fostered and required among teachers and students in order to accomplish learning goals. In these schools, 
technology use is not dependent on the teachers’ preferences, since the schools were designed with an educational 
technology focus. Therefore, educational technology is not just an additional tool to be used, but an important 
element of teaching and learning in these schools (Edutopia, 2010). 

With reference to the adequate and effective use of educational technology in schools that have undergone 
paradigm change, the industrial-age paradigm itself might be the biggest factor that contributes to limited and 
ineffective use of an LMS by teachers in traditional schools today. In addition, as mentioned in the results section, 
there is an emerging need for a more integrated and functional educational technology that can facilitate all of the 
information-age functions identified by Reigeluth et al. (2008). 

To this end, we see a great potential in Personalized Integrated Educational System (PIES) as a promising 
educational technology, which can meet the needs of students and teachers in the information age. PIES is a unique 
concept to define an educational technology which has all of the information-age functions identified by Reigeluth 
et al. (2008) with an open source architecture incorporating most recent emerging technologies including Web 2.0 
and online resources available in order to enhance and improve students’ learning. Further research needs to be 
conducted for the design and development of PIES in order to facilitate teachers’ use of technology systems more 
effectively and adequately. 
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The Role of Personalized Integrated Educational Systems in the Information- 

Age Paradigm of Education 



Sinem Aslan, Yeol Huh, Dabae Lee 
& 

Charles M. Reigeluth 
Indiana University 

Abstract 

Reigeluth, Watson, Watson, Dutta, Chen, and Powell (2008) identified major and secondary functions of technology 
for the information-age paradigm of education. Major functions include record keeping, planning, instruction, and 
assessment for student learning. Secondary functions are communication, general student data, school personnel 
information, and technology administration. Based on these functions, seven classroom teachers and one technology 
coordinator from three high schools were interviewed to reveal how they used technology systems with respect to 
these functions. The results revealed there were discrepancies between the current use of the systems and the ideal 
use for the information-age paradigm of education. Based on the results, recommendations are offered to teachers, 
policy makers and technology system designers for better meeting students’ information-age educational needs. 

Introduction 

A shift from the industrial age to the information age has happened (Toffler, 1980, 1990) and has been 
changing our social and economic lives more than we may have realized. The needs and expectations of students are 
very different than before. Similarly, the responsibilities and roles of teachers have been changing dramatically. 
Changes in the needs of students and teachers require changes in the supportive technologies that they use to meet 
their new needs. According to Gilhooly (2001) and Watson, Lee, and Reigeluth (2007), recent developments in 
educational technology can help meet these needs in the information-age paradigm of education. 

The information-age paradigm of education, which is currently implemented in very few schools, has a 
number of characteristics. First, it requires criterion-referenced assessment, which involves the assessment of 
student learning compared to what the student should attain or can handle with regards to the learning objectives 
(Knight, 2001), instead of ranking, ordering and comparing individuals with their classmates, which is an industrial- 
age practice (Smith, 1973). In addition, attainment-based student progress is essential in an information-age learning 
environment. According to Reigeluth (1995), achievement needs to be a “constant” at the competency level, but 
time should vary based on the progression of each individual student. This competency-based progression 
accommodates students’ different paces of learning. 

In addition, customized learning is another important characteristic of the information-age paradigm of 
education. In order to customize students’ learning, individual data such as a student’s goals, objectives, 
characteristics, and attainments mastered need to be kept. In this sense, each student will have a number of 
individualized attainments that need to be met. Since students have different interests, attainments, learning styles, 
and paces of learning, the role of the teacher needs to change in order to accommodate the new learner needs. 

Instead of being the source of information and content, the teacher is responsible for coaching and facilitating 
individual students’ learning based on their personal learning plans in the information-age paradigm of education 
(Reigeluth, 1995). 

Thomas (2000) promotes project-based learning as a way of creating a realistic constructivist environment 
where students are actively involved in an authentic learning environment and intrinsically motivated. Therefore, 
project-based learning can be utilized in the classroom in order to help students attain individualized objectives by 
doing the projects that are aligned with their personal learning plans. The teacher plays a role of facilitating and 
coaching students’ learning rather than delivering content. 

Theoretical Framework 

So what functions should a technology system perform in order to meet learners’ needs and accommodate 
these changes in the emerging information-age paradigm of education? According to Reigeluth et al. (2008), 
administrators, teachers, and students need a system that can facilitate and enhance four major information-age 
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functions for student learning: record keeping, planning, instruction, and assessment. The system also needs to 
perform several secondary functions, such as communication, general student data handling, school personnel 
information handling, and technology administration (Reigeluth et al.). These functions and their sub-functions are 
highlighted in Table 1. 



Table 1. 

Four Major Functions & Secondary’ Functions of an Information-Age PIES Proposed by Reigeluth et al. (2008) 



Record-Keeping 


Planning 


Instruction 


Assessment 


Secondary 

Functions 


- Standards 


- Long Term Goals 


- Project Initiation 


- Presenting 


- Communication 


Inventory 






Authentic 






- Current Options & 


- Instruction 


Tasks 


- General Student 


- Personal 
Attainments 


Requirements 


- Project Support 


- Evaluating 


Data 


Inventory 


- Short-Term Goals 




Student 


- School Personnel 






- Instructional 


Performances 


Information 


- Personal 
Characteristic 


- Projects 


Development 


- Providing 


-PIES 


Inventory 


- Teams 




Immediate 

Feedback 


Administration 




- Roles 




- Certification of 






- Contracts 




Attainments 










- Developing 
Students’ 
Assessment 










- Improving 
Instruction 










& Assessment 





Watson et al. (2007) point out that the information-age paradigm of education requires a kind of computer 
system that has built-in components to support the information-age functions mentioned above. While the term 
Learning Management System (LMS) was initially used for this kind of information-age system, it was often 
confused with Course Management Systems (CMS) and Integrated Learning Systems (ILS), so the term 
Personalized Integrated Educational System (PIES) was adopted. 

Methods 



Research Question 

The major research question of this study is: 

1 . How are classroom teachers using their technology systems with regards to the information-age functions 
identified by Reigeluth et al. (2008): 

a. Major Functions 

b. Secondary Functions. 

Research Context and Design 

This research study was a qualitative case study with semi-structured interviews. It took place in all three 
high schools, which we call High School A, High School B, and High School C, in a relatively homogeneous small 
city in the Midwestern United States. The school district had approximately 10,000 students attending elementary. 
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middle, and high schools. In the three high schools teachers used technology systems relatively actively. At the time 
of this study (2008-09), High School A had approximately 1,600 students, High School B had about 1,700 students, 
and High School C had around 80 students (ninth and tenth grades only), but increasing one grade level each year 
until it includes 12th grade). Technology use and resources varied across the three high schools. For instance, the 
curriculum in High School C was highly technology-focused and team-project oriented. 

In these high schools, Moodle was the predominant technology system used to foster student learning. 
Therefore, the features and use of Moodle were investigated for this study, but another system, Skyward, was also 
investigated. Moodle is a free system that has open-source architecture. It can be used for developing, managing, 
and utilizing learning sites for students’ learning. It has a flexible interface, which allows teachers to customize it to 
meet their classes’ needs. Skyward is a proprietary system used only for administrative issues, such as storing and 
managing the data related to student management. It is widely used for K-12 settings, being used in over 1,300 
school districts around the world (Skyward, 2010). 

Participants 

The purposive sampling technique was used to select the participants from the three high schools. The 
criterion for selection was a high level of usage of Moodle to support student learning. A preliminary interview was 
conducted with the technology coordinator, who was responsible for managing Moodle in all three schools, to 
identify all teachers who met this criterion and also to get a general understanding of how teachers used Moodle in 
the school district. The technology coordinator identified eight classroom teachers from different subject areas and 
grade levels who actively used Moodle. While we would like to have had more participants, this was all who met 
the criterion in this school district’s high schools, and it provided a sufficient number for triangulation of results. 
Invitational e-mails were sent to all eight of them, and seven agreed to participate in this study, as well as the 
technology coordinator. Table 2 outlines general information about these participants, their schools, and their subject 
areas. 

Table 2 

Research Participants, School and Subject Area Information 



Participant pseudonyms 


High school 
pseudonyms 


Subject areas 


Cody (Male) 


- 


Technology Coordinator 


Amber (Female) 


A 


Math teacher 


Laurie (Female) 


A 


Japanese & ENL teacher 


Joshua (Male) 


A 


Math teacher 


Michael (Male) 


B 


Math teacher 


Clint (Male) 


B 


Language Arts 


Jake (Male) 


C 


American History teacher 


Angela (Female) 


C 


Software technology and careers 



Data Sources/Data Instruments 

Data were gathered from the seven classroom teachers and one technology coordinator through semi- 
structured interviews. These interviews took about an hour to an hour and half for each of the participants. All 
interviews were audio-recorded, and transcribed verbatim. The interview questions focused on gathering data about 
how teachers were using their technology systems, such as Moodle and Skyward, and whether these uses were 
aligned with the four major functions as well as the secondary functions of an information-age PIES. For member 
checking, seven of the eight participants reviewed the transcription of their interviews by email correspondences. 

The other participant was out of contact. 

Data Analysis 

The primary analysis method was content analysis. The coding sheet was prepared according to the 
theoretical framework of the PIES functions. Three researchers coded the transcribed data of all eight interviews 
independently using separate coding sheets for each interview in order to check inter-rater reliability. After 
completing individual coding, the three researchers discussed their results to reconcile their differences and created a 



7 




final coding sheet with a 100% level of agreement. Using this final coding sheet, the researchers analyzed the data 
and induced emerging themes. 

Results 



Results for Major Functions 

Based on our findings, Moodle was not used for the record keeping function. All of the seven teachers used 
Skyward to keep their students’ grades for each course, but they did not utilize it to keep track of personal 
attainments, personal characteristics, or standards for students. In addition, there was limited evidence that suggests 
Moodle was not utilized for creating customized learning plans. Two teachers out of seven suggested that there 
were two major factors that constrained them. First, the central office discouraged teachers from setting 
individualized learning goals. Second, teachers had not found any functions they could utilize for planning in 
Moodle or had not figured out how to use the functions. 

The third function of PIES is instruction , and it includes four sub-categories: project initiation, instruction, 
project support, and instructional development. Overall, it was found that a few interviewed teachers had used the 
instruction function to a small extent. For the first sub-function, project initiation, two out of seven teachers had used 
Moodle, however, their use of Moodle was very limited for this sub- function. In addition, evidence of using Moodle 
for other sub-functions of instruction such as project support and instructional development were almost entirely 
absent from the interviews. 

In addition, use of Moodle to assess students' academic achievement was limited to formative evaluation, 
which had been used by only three teachers. Only these three teachers had used the quiz function to provide 
feedback on students’ performance and had utilized the results to improve their instruction. However, none of them 
had used Moodle for summative assessment. 

Results for Secondary Functions 

Teachers used different tools to communicate with parents and students. They frequently used 
asynchronous chat to communicate with students instead of instant messaging and used Moodle to give feedback to 
the students. Our findings suggest that there was no use of Moodle for communication among teachers. 

The technology coordinator stated that in these three high schools, they used Moodle to record each of the 
students’ information to the database. However, these records did not include information related to the information- 
age components mentioned above. These records were initiated at the beginning of each school year, and then they 
were static during the school year. 

As general student data were not kept in terms of the learner-centered approach, school personnel 
information was just kept for documentation. There were no data gathered about administration of Moodle. The 
implications of these results are discussed in the paper. The study identified the following implications: 

• Discrepancies between the current use of Moodle and ideal information-age functions, 

• Different uses of technology and tools for different tasks (e.g. Skyward for keeping grades), 

• Different uses of Moodle in different subject areas. 

Conclusion & Discussion 

Research findings suggest that there were discrepancies between the current use of technology systems and 
the identified functions of PIES in the information age. None of the seven teachers utilized Moodle for keeping 
students’ personalized record. Also, there was no evidence that Moodle was being utilized for creating and storing a 
customized learning plan for each individual student. The interviews with the teachers indicated that the 
standardized education system discouraged teachers from setting individualized learning goals, and they could not 
locate enough features that supported individualized learning in Moodle. Although the teachers frequently used 
Moodle for sharing resources and instruction, and utilized the discussion-forum, chatting, and glossary features for 
instructional purposes, evidence of project support and instructional development were scarce or nonexistent. The 
quiz function was frequently used to formatively assess students’ knowledge, but the means of assessing students’ 
knowledge was far from that of the information-age learning paradigm. Moodle was partially used for secondary 
functions. General student data and school personnel data were kept in Moodle for documentation. Instead of 
Moodle, Skyward was used for one-way communication with students and parents, and email was used for two-way 
communication. 




In addition, it was found that the teachers used different types of technologies for different tasks. This was 
mainly because of the schools’ policies on the use of technology systems and the teachers’ preferences. The teachers 
had to use what the school district required. When they had freedom to choose, they used more familiar 
technologies. Also, teachers in different subject areas used different kinds of technologies. They identified useful 
features based on the characteristics of their subject content, and utilized those functions. For example, an American 
studies teacher and Japanese and ENL teacher frequently used the discussion forum and chatting functions. On the 
other hand, math and technology teachers found those irrelevant to their subject areas, and found other features that 
could be easily adopted for their subjects, such as the quiz function. 

Based on the findings, the following suggestions to teachers, technology system designers, and 
policymakers were made. Teachers need training both in instructional methods for customized, learner-centered 
instruction, and in use of a new technology system, and they need more time to develop instruction and tests during 
their first year of using a new system. In order to facilitate teachers’ uses of a technology system, technology 
system designers should make it easier to learn and use, make it interoperable with other technology systems, as 
well as incorporate more functions into their system tailored to the various subject areas. Educational policymakers 
need to find a way to better facilitate and support the customized, learner-centered educational paradigm, including 
providing funding for technology systems and teacher training, and letting teachers devote additional time to 
developing customized instruction and assessments. 
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Abstract 

This case study presents how Facebook, Blogger and Moodle were used in three different graduate classes 
in order to enhance and support the class interaction on the Internet. Using Facebook as a social space for the class, 
blogger as a teacher reflection tool, and Moodle as resource repository, assignment submission and grading 
platform, we looked for the affordances and limitations of each tool for different educational purposes. In this paper, 
we intend to describe our teaching strategies for incorporating these platforms into the classes. We also include 
instructor and student reflections on the use of these platforms. Finally we describe instructional benefits of each 
tool and conclude with the guidelines of using each one of them to support face-to-face and blended courses. 

Descriptors: social networking, online learning, distance education 



Introduction 

For more than a decade, educational institutions have been challenged to reconsider their current 
educational approaches in the light of an increasing popularity of social and networked Internet technologies. New 
media technologies have provided opportunities for users to create, share and express themselves in different media 
channels (Jenkins et at, 2006). Although educators are witnessing a shift in the notions of learning and teaching as 
shaped by social and cultural practices of social media and new digital technologies, traditional teacher-directed 
pedagogies still dominate online learning environments in the higher education context. Similarly, traditional 
learning management systems (LMS) have been widely used as course support systems to provide a place for 
assignment submission, grading and resource repository. The social interaction in these platforms generally has been 
limited to the text-based discussion forums where students and the instructors interact asynchronously. However, 
these platforms often lacked necessary tools and social structure to support student engagement and social 
connection (Dunlap & Lowenthal, 2009). It is argued that the educational philosophy that undergirds these systems 
guides the teachers to think and act within the defined traditional teaching paradigms lacking the flexibility to be 
creative and adoptive to different learning contexts (Lane, 2009). Moreover, the conversation that happens within 
these systems die after the semester ends and don’t exist outside of the classroom. 

To meet these challenges of traditional learning management systems, we have integrated social tools to 
our semester long graduate classes. Using Facebook as a social space for the class, blogger as a teacher reflection 
platform and Moodle as a resource repository, assignment submission, grading and discussion platform; we looked 
for potentials of each tool for our different educational purposes. This paper presents the design decisions behind the 
use of different teaching strategies with Facebook, Moodle and Blogger tools in three different course contexts. We 
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also describe the instructional benefits of each tool along with their limitations; and conclude with the guidelines of 
using each one of them to support the interaction in blended and online courses. 



Context 

This study took place in the context of two blended and one online graduate course (1) Using Technology 
in Learning and Teaching, (2) Technology and Teacher Education, (3) Technology Assisted Project Based Learning 
in Curriculum and Instructional Technology program at a large research university in the Midwestern USA. The 
courses were offered in the fall of 2009 and the spring of 2010. 

Using Technology in Learning and Teaching Course 

The purpose of this course was to acquaint the students with current educational applications of digital 
technologies in teaching and connections of these applications to contemporary learning theory. Emphasis was 
placed on both the "how" and "why" of using digital technology in classrooms. Integrating digital technology to 
improve instruction and create student-centered classrooms was the central theme of the course. Students were 
encouraged to connect current learning theory to the creation of digital technology classroom activities for students. 
The course was structured using a combination of resources, discussion and student presentations. The course was 
project-based and almost every week, since the beginning of the semester, students were asked to create a project 
(usually a learning activity) around a particular topic. Each week, students received an agenda for the week and an 
assignment due at the beginning of the following week in class. 

The topics covered in the course included: Overview/History of Digital Technology in Classrooms, Word 
Processing and the Teaching of Writing, Blogging for the Classroom, Web Resources for Classrooms, Logo and 
Experiential Learning, Problem Solving, Digital Images and Digital Story Telling, Wikis for the Classroom, 
Podcasting, Distance Education, Social Networking in Education, Ethics and Equity, Virtual Worlds and Second 
Life, Technology in Teacher Education and Visions for the Future. In addition to these topics and technologies 
introduced by the course instructors, each week of the course, students were asked to evaluate and present a 
technology resource that can be used in teaching and learning environments. 13 female and 2 male students were 
enrolled in this course. 

Technology and Teacher Education Course 

The purpose of the second course. Technology and Teacher Education, was to provide graduate students 
mentoring skills. Students mentored one or two faculty members to use new technologies or different strategies to 
improve their courses. Bandura (1977)'s social learning theory and Roger (2003)'s diffusion of innovations theory 
were elaborated during the semester and students were encouraged to think on these theories while describing their 
progress with their mentees. Students met with their mentees weekly and wrote all observation memos onto their 
blogs in Moodle. Students were also paired to give feedback to each other. 4 female and 2 male students were 
enrolled this course. They had various academic backgrounds such as elementary education, teaching English as a 
second language, archeology, Literacy education and instructional technology. 

Technology Assisted Project Based Learning Course 

The purpose of this interactive online course was to explore Project Based Learning (PBL) and the 
potential to use technology to enhance PBL for students and teachers. Course topics included an introduction to 
PBL procedures for creating PBL lessons and units, technology enhanced PBL, in-service ideas for PBL, research 
on PBL. As a culminating activity, students developed a PBL unit for use in their own teaching or training. 15 
graduate students were enrolled in the course. 



Data Sources 

Data sources for this case study include Moodle, Blogger and Facebook contents posted on each platform 
either in the form of text or media such as pictures and videos. Discussions, blog entries, wall posts, student and 
instructor comments were also used as data sources. Additionally, a survey was sent to the students after the grades 
were released at the end of the semester to collect information on their perspectives on the advantages, challenges, 
limitations and affordances of these platforms for the classes. 
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Data Analysis 



A qualitative data analysis was conducted to analyze and present the themes emerged from the analysis of 
various data sources. First, the researchers independently reviewed the data sources and defined the major areas of 
commonality and differences. During the next phase the researchers worked together to merge the themes into a 
consistent whole through a back-and-forth, part-to-whole process of interpretation (Spiggle, 1994). Researchers 
resolved the inconsistencies through discussion and reviewing the data together. Persistent observation, prolonged 
engagement, triangulation and peer debriefing were used to ensure the trustworthiness in the study. This study will 
provide a "thick description" (Lincoln & Guba, 1985) of a case that is presented here to provide "sufficient 
information about the context in which an inquiry is carried out so that anyone else interested in transferability has a 
base information appropriate to the judgment” (Lincoln & Guba, 1985, p. 124). 

Results and Conclusions 

Collaboration, anytime anywhere communication and content creation have become the main activities on 
the Internet where users actively engage as the producers of the information rather than being the passive 
consumers. Some educators have used these tools in the higher education institutions, yet there is still a limited 
research on the educational uses of these tools with the innovative teaching and learning activities. This paper is an 
attempt to present cases where these tools were used extensively in three courses along with the open source 
classroom management system. The purpose was to understand the potential of each platform and explore the 
affordances as well as limitations of their use to support classroom interaction, socialization, communication and 
course maintenance. Accordingly, the emergent themes are presented below: 

Providing a Private and Secure Online Environment with Moodle 

Moodle was used as a course management platform for the purpose of resource sharing, class discussions, 
student blogs and peer feedback on blogs. Course agenda and materials were posted on the weekly Moodle schedule 
each week. Moodle provided a hub for the courses where all the other social media tools were easily integrated 
through the RSS feeds. Having a private and secure environment, Moodle allowed for keeping student blogs and 
discussions throughout the semester. Subscription to the forum option was also used extensively by the students. 
Moreover, third party applications and gadgets were adopted such as glossaries, wiki, chat and Wimba tools. Since 
Moodle had just been integrated to the college system, students and instructors had to spend some time to 
understand the affordances and limitations of the platform. Different than the previous course management platform, 
Moodle provided a weekly course design format that allowed the course content to be presented in an organized 
fashion. One of the students commented: “I have never had any technical problems with Moodle like other online 
courses. This creates no stress or frustration and I can spend my time doing the work instead of trying to figure out 
the technology’ and why it does not work”. It seems clear that having less usability problems related to the course 
management platform, helped students easily adapt to the new interface. 

Faeebook as a Class Socialization Platform 

In addition to the Moodle, a class Faeebook group was created to enhance the class communication and 
resource sharing. Since learning new technologies played an important role in the course context, Faeebook was 
used extensively during the semester for posting current news and resources related to technology and education. 
Faeebook was also used as a platform for sharing the videos and pictures of the class. In the faculty-mentoring 
course, faculty mentees paired up with the students in this course were also invited to join to the Faeebook group in 
order to enhance the collaboration and communication between the mentors and the mentees. 

In the courses, Faeebook groups increased students’ interaction and involvement early in the semester, 
providing them a place to get to know each other, particularly in the online course. One of the students commented 
"lam hoping Faeebook will help me in my quest to learn everyone's name". Another indicated " Faeebook is much 
more informal and user friendly". Students and instructors also shared their pictures and news on the Faeebook wall. 
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Increasing In-class Communication with the Facebook Group Wall 

Facebook group wall provided students a space where they could post their questions about an issue after 
each class and get answers either from their peers or from the instructors. Facebook wall was frequently used during 
the semester with an increasing frequency towards the end of the semester. Students were asking their questions 
about the assignments, due dates, submissions and final paper topics and getting quick responses. When asked about 
their insights on Facebook usage in this class, one of the students commented, "I think that Facebook should 
continue to be used for links, resources, discussion and troubleshooting" . 

Online Discussions: Facebook or LMS? 

Since Facebook group application was not designed as a learning management system; it lacked some of 
the features that the students needed while discussing a topic in a discussion forum. Moreover Facebook group did 
not allow uploading word or pdf files nor had a grading or assignment submission features. Therefore for more 
structured discussions LMS was preferred. One student commented on using Facebook as a discussion platform: 

In terms of practicality, I do think the Facebook has a number of flaws, particularly dealing with 
discussions and wall posts. I have trouble remembering what information is posted in what area and it's 
really clunky’ when trying to reply to a particular post early in a discussion. I do think Facebook is 
convenient, though. I'm on it so often that it's easy to just hop onto the group page and browse around. 
Maybe we can use Facebook in a more focused way. Perhaps we can just use Facebook for resources that 
we find or something like that. Then we can have our discussions on the blog and use LMS for assignments, 
class information, and more formal academic kind of things. If we can just clarify the organization a bit 
better, then I think we can better utilize the three sites. 

This was a reply to the instructors’ blog post asking students to comment on the platforms used in the class 
and the problems they faced. Another student stated, "For facebook and LMS, I think they have the same function to 
some extent. I know it is easier and fast to post anything to Facebook but I can say that LMS is more effective for 
online discussions because we can reply anyone 's posting but on Facebook interaction is more limited because as 
we talked in the class our reply to somebody 's posting (for example, a post in the middle) seems at the end of the 
discussion which causes the interaction among members to be limited''. 

Facebook introduced a new design for the group application at the mid point of the semester. Within the 
new interface the students and the instructors could comment on each other’s wall posts and get notifications when 
the wall post received comments. From that point on Facebook Group wall was used more frequently especially for 
the class announcements and questions regarding to the class activities. 



Using Blogger as an Instructor Reflection Platform 

The course instructor used Blogger extensively in her previous courses to reflect on the class experiences. 
Being familiar with the platform, she setup a class blogger site and posted a blog post each week to write about her 
reflections on the class, issues that she felt remained unanswered and to share the pictures from the class session. 
She also gave feedback to the students who did presentations during that week. Each week her post received 
comments from the students addressing the issues in the class. One of the students commented on instructor’s blog " 
I think it is effective because you can get immediate feedback from students and having an archive at the end of the 
semester is a another good part of it". Another student commented, "to me the blog seems more personal and less 
confusing. The conversations are clear whereas on Facebook the conversations are posted all over. I find the blog 
easier to read and find myself reading it first and no so much Facebook. ” 



Conclusions, Implications and Future Works 

The results of this study indicate that the use of these tools are context dependent and requires careful 
analysis of learner’s characteristics and needs. Getting students’ input at the very beginning of the semester, the 
instructors devoted first week’s class on the design of these platforms with the students, asking them about their 
previous experiences, dislikes, likes, suggestions and critiques. Getting their feedback early on the semester and 
incorporating more feedback throughout the semester, forms an understanding that these platforms were created for 
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and with them to answer their needs. This way, students may take the ownership in these platforms and interact in a 
more active way. 

Using different tools in a class might be overwhelming for the students and the instructors. However since 
the context for the courses was technology in teaching and learning, students not only experienced the tools as 
students but also explored potentials for their classrooms as educators. Moreover instead of depending on a single 
tool, which may not support all the interaction and collaboration in a course, different tools can be incorporated into 
the classroom activities with a careful planning. For instance, instructors can use Facebook’s real time web 
capabilities for informing students about the updates and news on another platform. Additionaly, instead of solely 
depending on one platform, presenting students different tools may help them to be better prepared for the 
workplace where they would interact with variety of technologies. 

In conclusion, analysis of the observations and the students' opinions indicated that each platform has 
different affordances and limitations. Convenient communication platforms are easy to use but privacy issues and 
simplicity of the communication tools may create challenges. Traditional learning management systems may be easy 
to access for the instructors, yet they may not provide the collaborative and social features that social tools present. 
Newer and more dynamic learning management systems may have innovative features, but they may require a 
learning curve for the instructors and additional time for technical training if an institutional support is not provided. 
One of the students commented, “/ have enjoyed this week's online communication a lot. Having so many 
possibilities for interaction outside of the classroom is so good. ” Recognizing this potential, instructors may 
incorporate social tools into their classrooms with innovative uses of learning management systems to develop 
pedagogically sound and sustainable learning environments. 
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According to Dick, Carey and Carey, “the best framework for the design of a formative evaluation is the 
instructional strategy.” (2009, p. 259). The instructional strategy best informs the purpose of conducting a formative 
evaluation for discovering errors or inconsistencies in teaching and learning that are limiting student clarity and 
motivation and in turn impacting achievement. Within the formative evaluation framework there are several areas 
that can be evaluated including type of learning, content, clarity, motivation and management. (Dick, Carey & 
Carey, 2009, p. 260). This formative evaluation will investigate instructional clarity and student motivation and 
perception of course relevance as judged by the target learners and the instructors. 

Today most students enrolling in a required undergraduate computer literacy course have some previous 
experience using software to solve education and business problems. (Kalman, 2007, p. 24). The instructional 
strategies used to teach new software have remained virtually unchanged for the past twenty years. Strategies such 
as demonstration and practice have been proven to be effective in assisting students in developing new skills and 
improving previously developed skills however teacher-led demonstrations greatly reduce individualized student 
learning and become “very difficult to implement with heterogeneous skill sets and experience.” (Kalman, 2009, p. 
25). The purpose of this formative evaluation was to investigate the use and effectiveness of technology-based 
instructional design strategies related to student practice and assessment activities. These instructional design 
strategies, software tutorials and simulations, were intended to improve student practice and project selection and 
increase motivation and engagement as well as transfer of skills. 

As most Universities now require an undergraduate course in Computer Literacy, this formative evaluation 
contributes to building knowledge of the principles and instructional strategies being employed to provide students 
with opportunities to practice and learn using what they report as the best instructional strategies for building on 
their existing knowledge and continuing to improve their Computer Literacy. 

Based on the results of this formative evaluation, the course designers continued to make improvements to 
the course to better meet the needs of the wide-variety of undergraduate students who enroll. The recommendations 
were intended to assist designers of introductory and advanced educational technology courses. Other course 
creators that are seeking to help students be more computer literate in a variety of teaching and training settings 
could also benefit from this evaluation. The terminal goal of this evaluation was to assist course instructional 
designers to tailor course content, materials and instructional strategies to be attractive, useful and applied in 
academic and work scenarios by the target students. 

Program Description 

EDT 321 : Computer Literacy is an undergraduate course administered by the Educational Technology 
program at Arizona State University. Graduate teaching assistants serve as instructors for the 1 1 face-to-face and 6 
online sections of the course, serving nearly 500 students. The course underwent a significant redesign for the 
Spring 2010 semester. The core curriculum came from Pearson Education's myitlab products. These products 
include a textbook and access to an online learning management system, myitlab.com. myitlab.com features a 
library of content including presentation slides, simulation and file-based activities and a test question bank. 
Additional course curriculum was updated and adapted from past iterations of EDT 321. 

The course consists of three primary units: (1) Microsoft Word, (2) Microsoft PowerPoint, and (3) 
Microsoft Excel. Students completed training and projects within myitlab related to each of the three primary units. 
Students also completed three original unit projects in which they demonstrated their mastery of unit objectives. 
Students also completed a group final project incorporating Word, PowerPoint and Excel to create an effective 
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proposal for a fictitious educational technology grant. Course requirements not directly related to the three primary 
units included creating an e-portfolio and participating in educational technology related research studies. 



Evaluation Method 



Participants. 

Enrolled students participated in three phases of the evaluation: the attitude and perception phase, the 
open-ended focus group phase and the performance phase. A total of 280 students enrolled in EDT 321 in the 
Spring semester 2010 responded to an attitude and perception questionnaire. From the useable data we retrieved 236 
student responses, including 93 male (39%) and 143 (61%) females. The majority of them (92.4%) are under the age 
of 32. Seventy-eight students from the four randomly selected sections participated in guided open-ended 
discussion-board focus groups. A total of 94 students were administered three tests to provide further evidence of 
the effectiveness of the myitlab simulation tool to teach Microsoft Office skills. 

Other participants involved in the evaluation were nine course instructors who collectively taught 17 
sections of the course during Spring 2010. 

The current course students were a valuable source for measuring understanding of the course's strengths and 
weaknesses. Students described their technology skill level, satisfaction with the course and expectations of the 
course. The students also provided learning outcome data through scores on a multiple choice and two skill-based 
tests of the course objectives. 

The course instructors assisted with the implementation of the evaluation activities by facilitating 
discussions using the discussion board in Blackboard and providing students the link to the course evaluation 
survey. Instructors also served as a data source by participating in a modified version of the course evaluation 
survey. 



Attitude/perception questionnaire. 

A 13 -item questionnaire was made available through an online survey tool. The instructors of each EDT 
321 section provided the link to the questionnaire and explained its purpose during regular class time. By recruiting 
the instructors to administer the questionnaire during class time, the response rate was very high. Data from the 
attitude/perception questionnaire was used to provide data on course effectiveness and recommendations for course 
improvements in the Results and Discussion sections of this report. 

In-class discussion board focus groups. 

Using the discussion board function in Blackboard, instructors conducted a focus group activity during 
class time. Students responded to open-ended questions designed to elicit more detailed and personal experiences 
with the course and technology. The purpose of using the focus group format was to encourage student interactivity 
to illuminate unanticipated issues, expectations or needs. Again, participation rates were very high since the 
instructors administered the focus group activity during class. Student feedback provided in the focus group 
interview activity was used to provide data on course effectiveness and recommendations for course improvements 
in the Results and Discussion sections of this report. 

Performance. 

Student performance in EDT 321 is measured throughout the course through activities in myitlab that are 
scored automatically by the mitlab system and original student projects which are graded by instructors using 
rubrics. Because multiple attempts are allowed for the myitlab activities and projects are graded by multiple 
instructors, the grades provided on these artifacts were not consistent student performance measures. 
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Student performance was measured using a multiple choice knowledge test and two skill-based assessments 
delivered through a simulation in myitlab. The assessment questions were taken from the myitlab test bank to ensure 
objective/assessment alignment. The multiple-choice test contained 20-multiple choice questions to measure student 
understanding of the Microsoft Office features covered in the course. Each of the skill-based assessments included 
20 skills for students to complete in a simulated Microsoft Office environment. Student performance on these 
assessments was used to provide data on course effectiveness in the Results section of this report. 

Results 



Attitude/Perception Questionnaire Results 

A total of 280 students enrolled in EDT 321 in the spring semester 2010 responded to the course evaluation 
survey. Due to technical problems, we were able to retrieve data from 236 students’ complete responses. Among 
these 236 students, there were 93 male (39%) and 143 (61%) females. The majority of them (92.4%) were under the 
age of 32. 

This student composition reflects that EDT 321 is an undergraduate level course. One-sample chi-square 
test was conducted to assess whether the proportion of each class level was statistically different from .25, which 
assumes equal proportion of each class level. The results were non-significant, x 2 (3, N= 236) = 4.12 ,p = .25, 
indicating that the proportions of each undergraduate level were statistically equivalent. The effect size was .005, 
which was calculated using the formula effect size = T / (total sample size) * (degrees of freedom). We need to note 
that there were two graduate students reported in this survey. Considering these two graduate students only 
accounted for .8% of the total students, they were not inc luded in the one-sample chi-square test, as they were 
treated as outliers. 

In terms of enrolled students’ major, 116 majors were reported from the raw data entered by students on a 
survey open-response question. We divided them into two categories — education major and non-education majors 
which yielded 151 enrolled non-education majors students and 85 enrolled education major students. Our hypothesis 
was that the education major and non-education major students were equivalent in terms of proportions. One-sample 
chi-square test was conducted to assess this hypothesis. The results were significant, x 2 (1, N= 236) = 18.46,/? < 
.001. The effect size was .08. 

Examining students’ responses on the question “On which of the following types of software/technology 
have you received previous training (select all that apply)?,” students had most previous training experience on 
Word (179 responses accounting for 79% ), PowerPoint (153 responses accounting for 68%), and Microsoft Office 
Basics (147 responses accounting for 66%). In addition, students had previous training experience on Excel with 
110 responses accounting for 49%. Only a small percentage of the students had previous training on Web design and 
computer programming, with 17% and 11% respectively. 
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The following tables describe highlighted student responses from the questionnaire. 



Table 1 

Reasons Student Enrolled in EDT 321 



Reason 


Percentage 


Required Course 


81.7% 


Improve my 


42% 


technology skills 




Time course was 


23% 


offered 




Easy class 


16% 


3-hour block option 


9% 


Online 


31% 


Advisor’s 


19% 


Recommendation 




Computer Science 


25% 


General Studies 




Table 2 




Usefulness of Course Materials 



Material 


Not at All 
Useful 


Somewhat Useful, 
Useful or 
Very Useful 


Don’t Care 


Textbook 


29% 


70% 


.9% 


myitlab exercises 


8% 


92% 


0% 


myitlab graded 
projects 


5% 


95% 


0% 


Class discussion 


24.1% 


75.9% 


1.8% 


Research 

participation 


26.3% 


73.7% 


7.1% 


Customizing their 
project topics 


4.5% 


95.5% 


0% 


Retrieving 

information from the 
Internet 


12.9% 


87.1% 


3.6% 
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Table 3 

Student Attitudes toward Learning Opportunities 



Learning 


Percentage 


Percentage 




Opportunity 


Favoring 


Not Favoring 




MS Office at 
Intermediate to 
Advanced Level 


76% 


24% 




New software (i.e., 
Gmail or Google 
Docs) 


68% 


32% 




Internet as a learning 
tool 


65% 


35% 




Using software to 


45% 


55% 




analyze data 
Building websites 


61% 


39% 




Using graphic design 
software 


78% 


22% 




Working in groups to 
complete class 
assignments 


29% 


71% 




Playing video games 


31% 


69% 




Using Facebook 
safely and ethically 


40% 


42% 




Table 4 








Student Attitudes Toward Learning Materials 






Learning Materials 


Somewhat Useful, Useful Don’t Care 




and Very Useful 






Demonstration 

Videos 


82.4% 




3.2% 


Software simulations 


87.4% 




1.3% 


How-to books 


69.4% 




4.1% 


Hands-on activities 


96% 




.9% 


Projects 


97.2% 




0% 
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Table 5 

Student Scheduling Preferences 



Schedule 


Percentage 


Don’t Care 


19% 


One 3-hour session 
per week 


25% 


Two 90-min sessions 
per week 


45% 


Daytime 


34% 


Evening 


50% 


Focus Group Results 





Focus group interviews were facilitated by the course instructors using the Blackboard discussion tool. Two 
sections were randomly selected from the pool of six online sections. Two additional sections were randomly 
selected from the pool of eleven face-to-face classes. Seventy-eight students from the four randomly selected 
sections participated in the discussion-board focus groups. Listed below are the focus group questions with 
accompanying tables showing the top five student responses for each question. 


1. What is your favorite technology-related activity? (82 responses from 78 respondents) 


The most popular activities are the online activities of Social Networking and Web Browsing (30%). Students 
specifically noted Facebook and online photo sharing applications for the Social Networking category. The Web 
Browsing category includes specific responses such as Google Maps, news sites and search engine usage. Video 
Game responses include World of Warcraft, XBOX 360 and Wii. 


Table 6 

Favorite Technology Activity 




Activity 


Percentage 


Social Networking 


15% 


Web Browsing 


15% 


Video Games 


12% 


Graphic Design 


11% 


Mobile 


9% 


Other 


39% 



2. What is a new technology or software skill you would like to learn? (75 responses from 78 respondents) 

Students would like to learn Graphic Design skills (33%) more than any other software skill. The graphic design 
program most often noted by students was Adobe Photoshop. Productivity software at 19% includes office 
productivity software like Microsoft Office 2007 and Google Docs. Students want the skills to design websites 
(17%) but did not name a specific software package they would like to use. 



20 



Table 7 

Software Skill Desired 



Software Skill 


Percentage 


Graphic Design 


33% 


Productivity 


19% 


Web Design 


17% 


Video Editing 


11% 


Programming 


7% 


Other 


13% 



3. How would you suggest to a friend they learn a new technology or skill? (177 responses from 78 
respondents) 


Traditional classroom instruction (30%) was the most suggested method of instructions followed by online 
tutorials (20%). Students suggested community college courses, private training courses as well as university 
courses for the Class category. Students mentioned a variety of free online tutorials on sites like YouTube. Fifteen 
percent suggested using a book to learn a new technology or skill. In the Self-Taught category (12%) students 
recommended practicing with the technology without the aid of formal instruction or instructional materials. 


Table 8 

Suggested Learning Methods 




Learning Method 


Percentage 


Class 


30% 


Online Tutorial 


20% 


Book 


15% 


Self-Taught 


12% 


Ask a Friend 


11% 


Other 


12% 



4. Describe any positive experiences with the content and materials (i.e., textbook and/or myitlab) you've had 
participating in EDT 321. 

(65 responses from 78 respondents) 

Myitlab accounted for (37%) of reported positive experiences. Unit Content (29%) refers to positive 
experiences with specific unit content in Word, PowerPoint or Excel. Fourteen percent of students reported a 
positive experience with the course textbook. 
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Table 8 



Positive Experiences with Content and Materials 



C ontent/Material 


Percentage 


Myitlab 


37% 


Unit Content 


29% 


Textbook 


14% 


Course 


11% 


Other 


6% 



5. Describe your experience with each of the following modules prior to taking this course. Use beginner, 
intermediate and advanced and describe why you have chosen to classify yourself that way. (112 responses 
from 78 respondents) 

Only 8% of students reported their experience level with Word as beginner. Thirty-four percent of students 
described their experience with PowerPoint as beginner. More students reported themselves as beginners in Excel 
(70%) than the other two programs combined. 

Table 9 



Experience With Modules 



Software 


Experience Level 


Percentage 


Word 


Beginner 


8% 




Intermediate 


59% 




Advanced 


32% 


PowerPoint 


Beginner 


34% 




Intermediate 


53% 




Advanced 


13% 


Excel 


Beginner 


70% 




Intermediate 


24% 




Advanced 


5% 



6. Describe the most useful knowledge you have gained by participating in this course. (46 responses from 78 
respondents) 

Despite a majority of students reporting intermediate to advanced experience levels with Word and PowerPoint, 
they reported that the most useful knowledge they gained in the course was from PowerPoint and Word (56% 
combined). 
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Table 10 

Most Useful Knowledge Gained 



PowerPoint 


28% 


Word 


28% 


Office 2007 


9% 


Course 


4% 


Other 


26% 



Performance Results 

A total of 94 students were administered three tests to provide further evidence of the effectiveness of the 
myitlab simulation tool to teach Microsoft Office skills. The three sets of tests were one knowledge test (mean 
percentage 68.76%) with 20 multiple-choice questions, and two skill-based tests — skill based test I (mean 
percentage 66.48%) and skill based test II (mean percentage 51.59%) — with 20 tasks for each test. One student’s 
scores on all three tests were missing and an additional 12 students had missing scores on one or the other skill- 
based tests. Considering these 12 students had some knowledge on computer literacy that was reflected from their 
scores on knowledge check test, mean imputation was used to estimate the value of these missing scores. 

In order to reveal relationships between the students’ scores on the three tests, a scatterplot matrix was 
examined and linear patterns were identified. Person product-moment correlation coefficients were computed among 
the students’ scores on the three tests. To control for type I error, alpha was set at .017 (.05/3). Students’ scores on 
knowledge check test were significantly correlated to their scores on the skill based test I ,R = .42, p <.001 and 
significantly correlated to their scores on skill based test II, R = .34, <.001. This indicated that students who scored 
high on the knowledge test tended to scored high on the two skill based tests. In addition, we find the two skill based 
tests were also significantly correlated, R = .61, p <.001, indicating that students scored high on one skill based test 
tended to scored high on the other skill based test. 

Instructor Questionnaire 

Nine instructors were invited to participate an online instructor survey and among them, five (25 to 42 
years old) completed the survey. They had two to ten years of teaching experience at university level and all had 
previous training in MS Office. They all had a positive attitude of usefulness towards using the textbook, 
completing the myitlab activities, participating in class discussions and research, customizing project topics and 
retrieving online information. In addition, they all believed that demonstration videos, software simulations, hands- 
on books/activities and projects were helpful to student learning. 

Discussion 

In response to the terminal goals of the evaluation, the evaluators offer the following recommendations for 
improving EDT 321 as well as other Educational Technology courses directed at a similar audience. 

Course Enrollment Demographic Recommendations 

Students were surveyed why they decided to enroll in this course. From the survey data, this is very clear 
that the reason is that is course is a general studies requirement. This reflects that not only undergraduate advisors in 
College of Education advise their students to take this course, but also advisors in other colleges suggest students 
take EDT 321. This course enrollment demographic information informs instructional designers to include 
additional topics or skills to EDT 321, which will be useful for junior and senior students who will soon be seeking 
jobs. 
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Course Content 



Undergraduate students live in a hyper-connected world as evidenced by their two favorite technology- 
related activities, social networking and web browsing. In a traditional university course setting, it is especially 
pressing to keep a course relevant to student’s experience and expectations. Arizona State University maintains a 
Facebook application available only to ASU students. Along with the standard social connectivity functionality, the 
ASU Facebook application hosts course pages for every ASU course. These course pages include rosters, a course 
wall, polls and the ability to add teaching assistants. Integrating the ASU Facebook application into the content of 
EDT 321 will show students that the course is current and relevant. ASU has provided a FAQ for students a 
information page and guide for instructors and an introduction video. These resources will help instructors and 
teaching assistants to leverage the ASU Facebook application effectively. 

Students most want to learn graphic design skills and software (Adobe Photoshop). Though graphic design 
is not within the scope of EDT 321, this data and EDT 321 ’s large enrollment are significant assets to the Education 
Technology program. The program can leverage the positive experiences students had in EDT 321 by marketing 
EDT offerings throughout the duration of the semester in an effort to increase enrollment in courses like EDT 323. 
These marketing messages should be built into the EDT 321 content and delivered regularly by the teaching 
assistants. Based on this data, we fully support the graphic design-focused redesign of EDT 323. We see further 
opportunity to use the data and EDT 321’s enrollment to create new courses that meet student’s needs and increase 
enrollment for undergraduate and graduate-level Educational Technology courses, certificates and programs. 

The survey and focus group responses indicate that EDT 321 students want to learn more of the advanced 
skills of MS Office, how to access and use Google docs, more about building websites and using graphic design 
software. Not many students like working in groups and playing video games. This shows the need for 
undergraduate university courses to teach students how to communicate and collaborate with people and perhaps 
introduce some introductory project management skills. Considering most of the students are females and not many 
people consider playing video games as learning activity, the response pattern is not surprising since usually more 
males than females are interested in playing video games. We recommend that both EDT 321 and EDT 180 
incorporate some hands-on activities that emphasize the value of using simulations and video games for learning and 
educational research. 

Based on the data, the decision to adopt Pearson Education’s myitlab curriculum was a good one. Students 
provided overwhelmingly positive responses to the myitlab website and content. One student said, “myitlab 
exercises really help to understand the program functions and apply them to real world work.” Another said, “I think 
that MylTLab has worked extremely well for me. I learn best by example and hands-on experience, and the 
MylTLab set up perfectly suits my style of learning. The textbook works perfectly well with the MylTLab material 
and has become a very handy reference that I know I will continue to use long after the course is over.” We 
recommend the continued use of the myitlab website and textbook. 

We were surprised by student’s reported experience level with PowerPoint. As computer literacy course 
evaluators and instructors, a popular assumption about students is that they are proficient in Word and PowerPoint 
skills and deficient in Excel skills. While a portion of that assumption was proven to be correct; students reported 
beginner level experience for Word and Excel of 8% and 70%. 34% of students responded that they were 
PowerPoint beginners. Further, 28% of students responded that PowerPoint knowledge was the most useful they 
gained by participation in EDT 321. One student said, “I now feel very comfortable with PowerPoint. 1 am sure I 
will be able to use the knowledge I have gained to do things at work that others cannot.” Another said, “I have had a 
lot of exposure and experience with those Office modules. I signed up for this course because it satisfies a degree 
requirement. I also went into it thinking I would learn nothing new. I am extremely surprised how many new things 
I have learned to do with Word and Powerpoint.” 

Word tied for the most popular useful knowledge response, also at 28%. Based on student’s responses, 
they feel that Word and PowerPoint skills are the most useful because those skills are used regularly and valued in 
the workplace and the classroom. An educator’s tendency is to place emphasis on skills that students do not know, in 
this case, Excel. The data shows that students both need and value Word and PowerPoint skills. Currently each unit 



24 



of Word, PowerPoint and Excel are given equal time in the course. We suggest that the Word and PowerPoint units 
be given more time in future iterations of EDT 321. 

Course Materials 

The result of student attitudes towards the usefulness of some course components showed a general pattern 
that indicates that the current course materials are useful. Many students demonstrated favorable attitudes toward the 
textbook as a reference. But we cannot ignore that about 1/5 of students said that the textbook was not useful. The 
majority of students find the simulation myitlab useful. Therefore, we suggest keeping this component in the future 
and perhaps integrate it into the EDT 180 course but use different learning activities and assessments to differentiate 
the two courses. 

Considerations for Future Evaluation Activities 

There are several other course considerations that emerged in the open-ended survey questions and focus 
group interviews that were outside the scope of this evaluation project but that are noteworthy for instructional 
designers and course supervisors to consider in future design and evaluation projects designed to analyze this and 
other similar course audiences. We have summarized these considerations in Top Ten lists intended to summarize 
this information for further study. 

Top Ten Things Students Liked Best about EDT 321 

1 . Hands-on learning 

2. Choosing their own project topics 

3 . Online course option 

4. Guided practice in myitlab 

5. Enthusiastic, engaging instructor 

6. Instructor that responds in a timely way to my emails 

7. myitlab practice and grader projects 

8. Appropriate pace that allows me to manage my time to complete projects 

9. Learning new tips and tricks about Microsoft Office 

10. Helps me now in school and is preparing me for future work opportunities 

Top Ten Things Students Feel Could Be Improved about EDT 321 

1 . Wider variety of activities 

2. More engaging instructor presentations; keep class focused rather than everyone using Facebook, etc. 

3. Make the research requirement more clear and be sure there are enough studies 

4. The textbook should be optional since it is expensive 

5. Be sure information is posted in a consistent place; some is on Blackboard and other stuff is on myitlab 

6. It would be cool if the class was more Mac friendly 

7. Course should be online or in hybrid format 

8. Change discussion board frequency; weekly is too often 

9. Be sure computers work properly 

10. Clearer and more explanation of projects; show examples 
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Abstract: 

In light of the fact that several studies indicate that students can benefit from deeper understandings of the processes 
by which historical accounts are constructed, history educators have increasingly been focused on finding ways to 
teach students how to read and reason about events in the same manner as professional historians (Wineburg, 200 1 ; 
Spoehr & Spoehr, 1994; Hynd, Holschuh, & Hubbard, 2004; Wiley & Voss, 1996). One possible resource for 
supporting this development may come out of emerging web-based technologies. New technologies and increased 
access to historical records and artifacts posted the Internet may be precisely the tools that can help students (Bass, 
Rosenzweig, & Mason, 1999). Given the right context, we believe it is possible to combine such resources and tools 
to create an environment for students that could strengthen their abilities to read and reason about historical events. 
Moreover, we believe that social media, specifically, microblogging (Nardi, Schiano, Gumbrecht, & Swartz, 2004) 
could play a key role. 

Twitter is a micro-blogging service that enables a person to broadcast ideas or events to anyone following. 
TwHistory (http://twhistory.com) adapts this communication stream by using it to represent historical figures based 
on excerpts from journals, letters, or other primary source documents. By consulting multiple primaiy sources, the 
range of viewpoints, biases, experiences, and knowledge of people at the time will be represented, creating a more 
reliable secondary source (Barton, 2005). Twitter provides the basic elements for a recreating a historical event: 
individual profiles, communication, relationships, and time. Followers of Twitter reenactments get updates in real- 
time as the historical figures represented in a particular reenactment “communicate” by sending tweets, or Twitter 
messages, about what is happening. The 140-character messages are sent as close as possible to the time and day of 
the actual event and are shifted to the present tense to give the sense that they are happening in real-time. Here we 
outline the basic elements for how to set up a TwHistory > historical reenactment. 

Participants are tasked with representing a historical figure during a reenactment. While this may not align 
perfectly with the practice of historians, it does align well with the practice of history (e.g. living history museums). 
Participants are asked to research and then deliver a role as a historical figure. This is a complex problem because it 
usually involves identifying relevant historical documents and evaluating evidence that may not always be 
consistent. Participants may need to research the person’s career, develop and utilize information literacy and 
writing skills, and collaborate with other groups. 

Reenactment Recommendations 

Turner (1985) suggests guidelines for selecting an event to reenact: events related to cultural or national 
heritage and traditions, how easily and faithfully it may be reproduced, a clear and logical sequence, distinguishable 
characters, and dramatic impact. Turner also points out that a segment of a long-term event is much more 
manageable, but the larger it is, the more students will understand its significance. Once the event has been chosen, 
the TwHistory’ development framework can be divided into four steps: 1) role assignment, 2) content creation, 3) 
content sequencing, and 4) deployment. 

First, participants organize themselves and identify key historical figures. The facilitator has some 
flexibility at this point as to the extent of scaffolding provided to participants carrying out the reenactment. They 
may provide a set of documents and ask them to work within those sources. This lessens the burden on participants 
so they do not have to actually locate primary courses. Barton (2005) points out, however, that an actual historian 
would search out his or her own set of documents or evidence from which to work. Depending on the amount of 
time available, it may be worth the extra time to allow for locating the sources, rather than just analyzing previously 
selected ones. 
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Once the cast of historical characters is set, participants decide who will research whom. It may be 
beneficial to employ smaller groups of 3-4 in order to provide more characters for the reenactment while still 
promoting discussion about where to search for appropriate information on the figure, reflection on the quality and 
relevance of the information found, and group decision making about how to incorporate the information into the 
reenactment. 

In the second step, the content is researched, evaluated, and discussed. This step is where the bulk of the 
work as historian happens. Once the group is in agreement about what to portray and how, the tweets are written 
and scheduled to be sent at a specific time during the reenactment. Each historical figure should have a Twitter 
account with a username that conveys who the character is. Each character's tweets should represent that historical 
figure as accurately as possible, based on the available historical evidence. 

Within a classroom setting or other face to face environments, most of the coordination of research will 
happen in person. Thus, it would be useful to organize roles within the team, with a moderator to direct discussion 
and a scribe to record decisions as they are made. When recording the tweets, a spreadsheet should be used with 
columns to record information such as the date and time a message is to be posted, the message itself, a formula that 
calculates the length of a message to ensure it is not too long, and an indication that a message has been approved by 
the group. The spreadsheet will also facilitate importing and exporting data later when messages from multiple 
accounts are combined together. 

For reenactments where the participants are communicating online rather than in person, the same 
spreadsheet structure should be used. The moderator role is not likely to be necessary because of the asynchronous 
nature of online communication; however, one may be needed to keep participants motivated or to keep discussions 
on topic. The need for a scribe would likewise be lessened, as all participants would be writing out their thoughts 
and contributions already. Online, freely available collaboration tools such as Google Docs and Google Groups can 
facilitate discussion and the recording of decisions made. 

The third step involves combining all the character tweets together in chronological order. This provides an 
opportunity for participants to verify that all have written appropriate tweets that fit properly with the other Twitter 
characters. It is not necessary that all characters tweet the same event at the same time; depending on the situation, 
certain characters might not find out about an event until later. During the Gettysburg reenactment, one of the 
characters was a newspaper. Comparing the timeline of reports in the newspaper and in participant journals, it was 
obvious that the newspaper was often 2 or 3 weeks behind in publishing the news, which makes sense given the 
slower communication of the time. A decision was made to move up the newspaper reports to occur at the same 
time as they actually happened; this decreased the level of historical accuracy and lost some context about how news 
was disseminated at the time but made it easier for followers of the reenactment. It came down to a discussion and 
decision as a group; a different group might choose to make a different decision based on the goals of the 
participants in the reenactment. 

In the fourth step, the Twitter messages are scheduled with a timer program so that each tweet is sent at the 
appropriate times (Figure 1). The idea is to match the date and time of the events as closely as possible to when they 
actually occurred and to when other characters will be tweeting the same events. Using a timer program makes it 
possible to schedule tweets to happen closer to the real event time, without requiring students to manually post 
messages, especially if they are not always somewhere with internet access. For a smaller reenactment with a 
limited number of participants, it may not be necessary. A timer program is essential for a larger scale reenactment, 
especially one with important detailed timing from multiple participants that needs to be maintained. It may also be 
necessary in a classroom situation where certain websites at the school may be blocked. 
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Harpers_Weekly Gen. Meade's troops were placed in order, 
and charged down the hill and into the town. 

7:13 PM Jul 3rd from Tweet Master 




AD_Betts Col. Bennett wounded. Lieut. Connell, of Co. G., 
killed. 

5:58 PM Jul 3rd from Tweet Master 




Col_wainwright Some of my men rode to where the battle 
took place. They tell me the ground is covered with dead. 

5:23 PM Jul 3rd from Tweet Master 



Figure 1 - Example Twitter stream from the Battle of Gettysburg 

To prove the concept, a small group of volunteers developed TwHistory and “tweeted” the Battle of 
Gettysburg using journals and letters from fifteen soldiers present at the battle, starting about three months before 
and chronicling how they arrived in southern Pennsylvania and participated in the battle. The experiment generated 
many followers, and interest in the project grew. When one of the characters died on the first day of the virtual battle 
of Gettysburg, many followers retweeted this event, and that message brought additional followers as they became 
aware of the reenactment. 

Once the virtual battle had gone viral, a diverse set of followers subscribed to the Twitter feeds. One of the 
Gettysburg followers was Carla Federman, a high school teacher in the American Midwest. She adapted the 
Gettysburg model as part of her Cold War Flistory course. In that class, students re-enacted the Cuban Missile Crisis. 

Classroom use of TwHistory 

The Cuban Missile Crisis Twitter reenactment was the first case of TwHistory use in the classroom. Mrs. 
Federman organized her class into small groups, with each group responsible for representing a particular historical 
figure in the Crisis. During the two-week activity, students were given class time to research relevant sources, 
including primary sources from the Library of Congress website. Feedback from this preliminary classroom 
implementation was positive, with the teacher reporting student engagement and an interest to do another TwHistory 
reenactment in the future. 

Part of learning to think historically involves seeking to understand the context in which historical events 
take place. Although Mrs. Federman encouraged her Cold War History students to stay in character with their 
Twitter messages, they did not always do so. At one point in the reenactment Khrushchev sends a message after 
conducting a nuclear test and exclaims, “Boo-yah!” This highlights the potential need for peer review and 
assessment in future implementations of TwHistory. 

Given the flexible nature of the TwHistory’ framework and the diverse classroom settings in which it may 
be utilized, it is not imperative that all reenactments be structured as a precise reenactment. While it is generally 
recommended that text be quoted as accurately as possible, there are cases where that is not possible or 
recommended. Twitter itself limits what can be said to 140 characters. To work around this issue, researchers will 
sometimes break a longer message up into several consecutive tweets, but the most common practice in this case is 
that researchers will simply consolidate, paraphrase, and bring out the essence of what was said. During a 
reenactment of a trek across the Old West, it was found that some of the journals contained very poor spelling. An 
effort was made to keep the spelling and wording the same, but many words were fixed so followers would 
understand better what was being said. 

Another technique that a facilitator or teacher may choose to employ is to have students translate antiquated 
or formal language into current language or slang. The downside to this is decreased historical accuracy and context 
for followers of the reenactment. A benefit may be for the students creating the reenactment; being able to translate 
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into modem text requires that the student take the time to really understand what is being said. Thus, while 
Khrushchev’s out of character exclamation may be problematic for reenactment followers, it shows a level of 
understanding and motivation to participate on the part of the students who created it. 

Instructional Effectiveness 

In Merrill’s (2009) First Principles of Instruction, several principles align with and support the TwHistory 
model for effective instructional design. One important component Merrill recommends is the use of peer critique 
and discussion. This is an essential component of TwHistory’, as a project of significant size is not feasible without 
coordination. Another recommendation of Merrill is to include a component of public demonstration. The nature of 
Twitter is that it is public to the world. While the Twitter is just a tool, and a reenactment could be done privately on 
another type of network, keeping it public motivates students to put on a good show for those following them, yet 
with minimal privacy concerns, since student personal information is not being publically released. While the 
Khrushchev outburst example has served as a good example of a bad example, if the students involved in sneaking 
that message through had thought more about the fact that the world was watching, they may have showed more 
restraint. 



The central principle of Merrill’s is that instruction should be task centered. By focusing on a large scale 
project of building a historical reenactment, everything students do is evaluated as to how it contributes to that task. 
In the end, the success or failure of the reenactment can be directly tied to how well students worked together on the 
task. 

Conclusion 

Preliminary feedback from Mrs. Federman suggests potential gains in engagement and understanding, but a 
more carefully monitored implementation is yet to be explored. Future classroom implementations are planned to 
explore and refine historical thinking outcomes through student-created virtual reenactments. Of particular interest is 
the decision-making process used in prioritizing, contextualizing, and portraying specific people, moments, and 
ideas within a larger reenactment. Activities like TwHistory’ are promising and worthy of study because they not 
only offer an engaging opportunity for students to begin to leam the skills of a historian, but they also help students 
understand the often subjective nature of the historical accounts presented to them in textbooks and other 
instructional media. 
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Abstract 

One of the more recent trends in education is the use of smartphones for mobile learning. This paper 
provides an overview of mobile learning, smartphones, and the benefits and challenges that accompany the use of 
smartphones for mobile learning. The study involves an analysis of educational applications on the Blackberry, 
iPhone, and Android systems. Findings suggest that although the highest rated applications on smartphones are 
games and entertainment applications, educational applications can be location aware, collaborative and ‘interesting’ 
applications. 



Introduction 

What can be considered a mobile device? What is mobile learning? These are important questions to ask 
and pursue to solidify mobile learning as a discipline. Mobile phones, specifically smartphones, are devices that are 
used in defending the argument for the potential of mobile learning. Mobile learning or mLearning can be defined as 
“the intersection of mobile computing and eLearning (i.e., electronic learning): accessible resources wherever you 
are, strong search capabilities, rich interaction, powerful support for effective learning, and performance-based 
assessment” (Quinn, 2000, p. 1). mLearning can occur through mobile computational devices, such as: Palms, 
Windows CE machines, and mobile phones (Quinn, 2000). In particular, applications (i.e., apps) developed for 
smartphones are used as a method to deliver mLearning content. 

What is a smartphone? 

Types of smartphones include the iPhone, Blackberry, HTC Evo 4G, Samsung Epic 4G, Driod, and T- 
Mobile G2. Market research shows that BlackBerry smartphones lead the market share with 41.6%; Apple’s iPhone 
follows at 25.3%, but the iPhone market is growing (Lipsman, 2010). The Android has recently doubled its market 
share, and according to ChangeWave’s survey, out of 4,068 consumers, 21% of those planning to buy a smart phone 
in the next 90 days say they would prefer to have the Android operating system on their new phone (Carton, 2010). 

Smartphones have all of the capabilities of a normal cell phone, but also advanced features that can be 
found on a personal digital assistant or a computer, such as: e-mail, Internet, and web browsing (Cassavoy, 2010). 
Other significant features that further distinguish smartphones from regular cellphones are the ability to interact with 
laptops and computers, as well the installation of additional applications that are developed by the manufacturer, 
mobile phone company operator, or third party software developers (Singh, Bhargava, and Kain, 2008). Smartphone 
applications allow designers and developers to create tools for on demand learning. 

Benefits of mLearning and smartphones 

There are many observed benefits and affordances of mLearning, including providing on demand learning 
and improving communication and collaboration on projects (Schuler, 2009a). Smartphone applications allow 
designers and developers to create tools that can be used anywhere. Additional benefits include: 

1 . Compactable — Most smartphones fit in an adult sized hand. The use of applications has the potential to 
transform traditional classrooms by enhancing current techniques and allowing access to learning from 
your pocket. 

2. Portable — Due to the size and low weight, smartphones are highly transportable. 
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3 . 



Access to Rich Internet Applications (RIAs) — Smartphones enable the user to experience desktop 
computer-like applications without the constraints of static locality. 

4. Various capabilities — Most smartphones include a built in camera, video, and audio capturing tool, as 
well as GPS (i.e., global positioning system). 

Studies on the affective factors surrounding the use of smartphones in education have yielded findings 
indicating that there are many factors (e.g., making learning personal) that add to a learner's motivation (Jones et al., 
2006; Issroff, Scanlon & Jones, 2007). Research shows that understanding the motivational aspects of using 
smartphones can assist in increasing motivation and retention within classrooms (Issroff, Scanlon & Jones, 2007). 
Smartphone applications offer benefits that can impact current educational practices, but there are also challenges 
that accompany them. 



Challenges 

Current theoretical frameworks are not always sufficient for studying the issues or helping explain the 
impact of mobile technologies in different settings and for different populations (Schuler 2009a). Although 
researchers, such as Sharpies, Taylor, and Vavoula, have proposed standards that can be used for formulating a 
mobile learning theory, there is no widely accepted theory for assessing learning through mobile devices (2005). As 
the use of mobile phones in education continues to grow, there are challenges that must be identified and addressed, 
to enable full integration of these devices into curriculum design. Some of these challenges include: 

• The difficulty in determining which applications are valid for educational purposes, because there is no 
industry standard and several applications are marketed as such to increase sales (Schuler, 2009a). 

• The variability of smartphone brands and models in the market which results in complications for teachers 
who need to make their learning materials work for the various phones students own (Schuler, 2009a). 
Teachers may run into compatibility and design issues across platforms. 

• The potential revolution in the educational field, similar to online music downloading and music industry; 
this will raise issues of quality, privacy, copyright and assessment of formal and informal learning 
(Sharpies 2007). 

Additional challenges that developers face are incompatibility issues with Learning Management Systems 
(LMS) and Content Management Systems (CMS). Many institutions rely on a LMS or a CMS for learning, but not 
all systems are optimized for smartphone access. Secondly, there is varying availability of bandwidth and signal; 
depending on the region, city, or the corner of the room (in some extreme situation) that a person is on, it is possible 
to not receive a signal. Lastly, there are many financial challenges. For example, smartphones are expensive! The 
average cost for the lowest end smartphone is $100 to $200. Additionally, smartphones require the user to purchase 
a data plan as well as the standard mobile phone plan. This could be cost prohibitive for many individuals. 

Study 

Research on educational applications has focused predominantly on the iPhone and the Apple iTunes store. 
In a content analysis of the educational applications available in Apple's iTunes store, Schuler (2009b) found that 
"there is a significant market for children’s educational apps, toddler/preschool apps are more prominent than those 
for older children, the top educational apps cover a variety of different subjects, with foreign language and literacy 
being the most popular categories, and children’s apps are significantly cheaper than adult-targeted Apps" (p. 6-8). 
While Schuler’s research provides evidence of the use of educational apps on the iPhone, what about the other 
leading markets: specifically Blackberry and Android? Also, what percentages of smartphone apps (i.e., smart apps) 
in these markets are education-focused? The following study was conducted in October 2010 to answer these 
questions. 



Method 

The study involved three sets of analyses. First, in Study 1, the Blackberry, iPhone, and Android app stores 
were studied to determine the number of educational applications in the top 100-200 based on two categories: top 
paid and top free. In Study 2, a deeper analysis of the Blackberry, iPhone, and Android app stores was conducted to 
find the percentages and breakdown of non-educational applications in the top 100-200 range. The range from 1 00- 
200 was useful because the Blackberry, iPhone, and Android app stores do not provide the same search results. To 
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fix this and standardize the findings, percentages are used. And finally, in Study 3, the educational apps discovered 
in the first two studies are analyzed using Patten et al.’s Functionality Framework (2006). 

Results 



Study 1 



Starting with Blackberry, the leader in market share, the results show that out of the top 100 paid and top 
100 free apps, there are no education apps, although an education category does exist (Blackberry, 2010). For the 
iPhone, the situation is different. Among the Top 200 paid apps in the iPhone app store, there are five education 
apps: 



43. My very first app ($0.99), 

51. Star Walk - 5 stars astronomy guide ($2.99), 

81. Ace Flashcard ($0.99), 

152. Cookie Doodle (0.99), and 
177. Wheels on the Bus ($0.99). 

(Apple, 2010) 

For the top 200 free apps in the iPhone app store, there is only one education app: 64. American Museum 
of Natural Flistory: Cosmic Discoveries (Apple, 2010). Unfortunately, there is no education category in the Android 
platform, so no results were found (Android, 2010). 

Study 2 



In Study 2, breakdown and percentage data were collected for categories of top applications in each app 
store. The results for each app store are listed in Figures 1-3. Results show that games and entertainment apps are 
the highest rated applications across all three platforms. 
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Figure 1. Top 100 Blackberry Paid and Free Applications. Each | mark represents one application (Blackberry, 

2010 ). 
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iPhone App Store 
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Figure 2. Top 70 Apple iPhone Paid and Free Applications. Each | mark represents one application (Android, 2010). 
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Figure 3. Top 34 Android Paid Applications and Top 134 Android Free Applications. Each | mark represents one 
application (Apple, 2010). 

Study 3 



There are no educational applications in Blackberry and Android stores; however, there are applications 
that have categories similar to education. One of the most noticeable is productivity. Productivity applications help 
to increase the ability to perform one’s job. Many of these applications are used in learning situations and 
environments. For this reason, they are included in the third study. In this study, the productivity and educational 
apps from Study 2 are evaluated using Patten et al.’s Functionality Framework (2006). The Functionality 
Framework evaluates educational applications according to seven categories: administration, referential, interactive, 
microworld, data collection, location aware, and collaboration. Each category builds on the previous category. 
Descriptions of each category are described as follows: 

1 . Administration - “applications in this category do not scaffold or support knowledge construction 
and merely replicate tools already available on traditional platforms” (Patten et al., 2006, p. 297). 

2. Reference - this includes applications that do not go beyond “information delivery” such as note 
taking or electronic book readers (p. 297). 
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3. Interactive - applications that are creative and enable user interaction but often are of “drill and 
test type aimed at encouraging memorization” (p. 297). 

4. Microworld - applications that enable users to “construct knowledge through experimentation in 
constrained models of real world domains” (p. 298). 

5. Data collection - applications enable users to collect and interact with scientific, reflective, or 
multimedia content 

6. Location aware - applications in this category use the data collection category features with the 
addition of a global positioning system (GPS) 

7. Collaboration - applications that enable “knowledge sharing” (p. 299). 

Although the categories are useful, Patten et al. (2006) suggest that the most interesting applications are 
those that “facilitate learners to look away from their screen in order to engage with their surroundings and peers” (p. 
299). For this reason, the evaluation process will involve categorizing the application according to the Functionality 
Framework and asking the ‘interesting’ question: Does the application facilitate learners to look away from the 
screen in order to engage with surroundings and peers? 

The analysis below involves the two educational applications and the productivity applications across all 
three platforms. For each application, a brief description is provided with platform and free/paid information, 
followed by a categorization according to the Functionality Framework and ‘interesting’ question. 

Education Applications: 

• Star Walk - astronomy and constellation guide (iPhone Paid) - Location aware; interesting? Yes! 
This application enables you to move your iPhone in relation to the sky and find constellations, 
which in turns facilitates your engagement with the sky! 

• My Very First App - matching game designed based on Eric Carle’s books (iPhone Paid); 
Reference or Interactive; not interesting 

Productivity Applications: 

• Lookout Mobile Security - Protect your phone with Free lookout (Blackberry Free) - 
Administration; not interesting 

• PrivacyStar - caller id, callblocking (Blackberry Free) - Administration; not interesting 

• Who Is It - LED light alerts for contacts (Blackberry Free) - Interactive; not interesting 

• BeJoose - phone memory optimizer (Blackberry Paid) - Administration; not interesting 

• Secret Diary - store diary entries on your phone (Blackberry Paid) - Reference; not interesting 

• StickyNote - put customized sticky notes on your home screen (Blackberry Paid) - Reference; not 
interesting 

• RepliGo Reader - read .pdf documents on the go, add bookmarks, hyperlinks, and text search 
(Blackberry Paid) - Reference; not interesting 

• Best Alarm Clock & Weather and Temperature (iPhone Paid) - Administration; not interesting 

• Emoji Plus - Best Emoticon Keyboard (iPhone Paid) - Reference or Interactive; not interesting 

• AK Notepad - write notes and share via SMS or email (Android Free) - Collaboration; not 
interesting 

• Voice Recorder - save data and send through gmail (Android Free) - Data collection; not 
interesting 

• Battery Widget - shows the exact battery level and charging status (Android Free) - 
Administration; not interesting 

• OI Notepad - shows a list of notes that you can create, edit, and send (Android Free) - Reference; 
not interesting 

• Astrid Task/Todo List - features tagging, reminders, adding to calendar (Android Free) - 
Interactive; not interesting 

• Note Everything - features text-, paint-, voice-, checklist- and photo-notes (Android Free) - Data 
collection; interesting? Yes! This application can enable you to use your Droid as tool that 
facilitates your reflection and engagement with your surroundings to a deeper level! 

• handyCalc - calculator with automatic suggestion and solving (Android Free) - Administration; 
not interesting 

• OI File manager - browse fdes on your SD card (Android Free) - Administration; not interesting 
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• Tag-ToDo-List - features easy task handling, drawing graphical notes, and recording audio 
(Android Free) - Data collection; not interesting 

• JETCET PDF - read .pdf documents on the go (Android Free) - Reference; not interesting 

• RepliGo Reader - read .pdf documents on the go, add bookmarks, hyperlinks, and text search 
(Android Paid) - Reference; not interesting 

Future Research 

Future research must apply the findings in this paper to other mobile devices and platforms, in particular, 
Apple’s iPad. Figure 4 provides breakdown and percentage data for the categories of top applications on the iPad. It 
is clear that the percentage of productivity applications on the iPad is much higher than were found in Study 2 (18% 
paid apps on iPad compared to 3% paid apps on the Android, Blackberry, and iPhone). Does this imply that the iPad 
is a better platform for educational applications? Future research must be completed to test this conjecture. 
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Figure 4. Top 80 Apple iPad Paid and Free Applications. Each | mark represents one application (Apple, 2010). 

Conclusion 

Cathy Burnett (2009) suggests, “it is those who have grown up with digital technology who may be most 
likely to understand the potential of digital affordances” (p. 115). We need to embrace technology and use it for 
education, especially as these children who use technology grow older. It is evident that mLearning is becoming a 
prominent topic for educators, but there is a lack of research conducted on the use of education applications on 
smartphones. This paper established a starting point to think about educational applications and help designers 
create ‘interesting’ applications - applications that “facilitate learners to look away from their screen in order to 
engage with their surroundings and peers” (Patten et al., 2006, p. 299). 
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Visualization can be defined as the conversion of information to a symbolic representation of a particular 
idea, concept, or data object. In simplified terms, McCormick (1987) explained visualization as the process of 
transforming data, information, and knowledge into graphic symbolic presentations. There are advantages in use of 
tools designed to make the transformation of text possible. First, visualization tools can be used to make information 
more accessible. Early proponents reported successful use of visualization retrieval tools to present bibliographic 
displays generated from hugh amounts of digital information. According to Wise, (Wise, et al, 1995) data sources 
become manageable and information retrieval efficient through organized graphic displays. 

Secondly, visualization permits perception at a higher level. Tools designed to create visualizations allow 
users to explore the symbolic structure of a text. Thinking processes are enhanced as users are interactively engaged 
in the reading process. Users may create images in order to identify relationships, main ideas, and patterns thus 
facilitating concept attainment and critical analyses of reading materials (Sadoski, & Paivio, 2001). Johnson- 
Glenberg examined the effects of question-writing and word association with visual imagery. The results of her 
study showed significant increase in reading comprehension scores for struggling readers (2005). These and other 
generative strategies suggest visualization tools can be used for ensuring recall and comprehension for readers of all 
ages (Hook & Borner, 2005; Mills, 2009; Wittrock, 1992). In more simple settings such as elementary school 
classrooms, concepts maps, pictographs, and word clouds can be used for making connections and predicting themes 
within expository text (Foote, 2009; Oliver, 2009). Most teachers have used pictographs and other representations 
for reading lessons however word clouds have only recently been used in K-12 classrooms. 

The website Wordle.net provides an easy to use tool for creating word clouds by both teachers and K-12 
students. In higher education, visualization tools accessible through Many Eyes provide interactive activities that 
include word clouds, tag clouds, and word trees, but with more complex textual information (Eisenberg, 2008; Hook 
& Borner, 2005; Viegas, Wattenberg, & Feinberg, 2009). Developed by designers in IBM labs, Wordle is the 
product of the creators for Many Eyes. Wordles produced by cut and paste of a selected reading passages result in a 
simplistic image from a complex narrative. Zhang refers to these visualizations as metaphors. Mental images can be 
formed and conceptual ideas more easily recognized, communicated, understood, and remembered (Zhang, 2008). 

Both Wordle.net and Many-Eyes.alphaworks.ibm.com are interactive open source tools, but only Many 
Eyes permits registered users to post commentary on any of the thousands of visualizations published for public 
viewing. Although text can be entered directly through the user’s keyboard, value is added to the use of these tools 
through the cut & paste process possible with digital information. The wealth of accessible online resources makes 
use of these tools easy and efficient for educators. 
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Purpose of this Study 



Strategies using tools such as Wordle, Many Eyes, and Tag Cloud provided promise for visualization tools 
to enhance reading within a variety of contexts and learning environments. Could use of visualization tools such as 
word cloud imagery enhance reading comprehension and help both graduate and undergraduate students understand 
complex ideas and abstract concepts? This paper is a report on findings in use of a variety of visualization tools to 
enhance reading for adult learners. 



Methods 

Two populations were examined for use of visualization tools to enhance reading comprehension and 
concept attainment for assigned reading in both graduate and undergraduate classes. The first data collection 
involved graduate students in a master’s level program for instructional technology. Visualization tools were used to 
analyze the contents of a job description for school technology specialists. A 500 word online reading from the state 
department of education was used as the source document for analysis. Using their assigned reading to copy and 
paste into Wordle.net, students generated visualizations representing the main functions of a technology facilitator. 

In small group discussion forums, students analyzed the keywords displayed in the visualization, predicting possible 
scenarios for the position of technology facilitator in a K- 12 school. After the online discussions were concluded, 
students submitted proposals for internship experiences at their local schools. After writing the first draft of the 
internship proposal and posting these to the online discussion forum, students participated in a second discussion 
forum to compare interpretations of job functions for technology facilitators. Following the second discussion, a 
final proposal was written and submitted to the course instructor for evaluation. The design of internship proposals 
provided the first data set for this study. These are discussed in the results section below. 

A second data collection involved pre-service teachers who received instruction in use of Phrase Net tools 
to visualize conceptual relationships in an assigned reading. Students in a traditional face-to-face Health Education 
methods class were given an online book chapter as assigned reading on the topic of learning and cognition. 
Following the reading of the book chapter, multiple choice tests were used to assess understanding of concepts 
discussed in the online book chapter as well as measures of confidence levels related to personal understanding. Test 
items included these areas: 1) concept attainment; 2) relationship between concepts with supporting ideas and 
examples; 3) relevancy to personal experiences. Mean scores for each of the measures were used to identify 
differences in pre and post use of the visualization tools. Mean scores for students’ self reported confidence levels 
for pre and post tests were used to identify differences in confidence levels when using the visualization tools. 

A third group included both graduate and undergraduate students who examined visualization tools for 
potential use in their K12 classrooms. Review of students’ discussion threads revealed creative ideas for 
instructional use of tools. Lesson ideas were submitted to the course instructors who categorized each activity by 
skill, instructional method, outcomes, and assessment of learning. A summary of selected activities are presented in 
the results and discussion of this paper. 



Results and Discussion 



Word Clouds to Analyze Job Descriptions. 

Interns in the MAEd in Instructional Technology used visualization tools to analyze content of internship 
proposals compared with similar visualizations produced using content from an online job description provided by 
the state of education. The tag cloud tool was used to calculate the presence and count of unique words in a 
document as well as statistical analysis of frequency and context of words and letter strings within the document. 
Interns compared output from their internship proposals with output generated from job descriptions accessed 
through the state department of education. The content of the job descriptions were dropped into Tag Cloud tools to 
produce visualizations that include word counts for all terms within the document. The interns used the output from 
their visualizations as a guide for discussions in an online forum. The visualizations resulted in a focused dialog with 
emphases in main functions for the job of technology integration specialist and technology facilitator. Qualitative 
evaluation of the students’ proposals showed a clear and consistent alignment with national standards for these 
positions in K-12 schools. Interns’ proposals from previous semesters did not reveal depth of understanding evident 
in this pilot group using visualization tools to analyze future job descriptions. Keywords predominate in the 
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visualizations prompted interns to plan around terms such as “teacher support”, “instruction”; and “collaboration”. 
These were then used as prompts for negotiating projects for the semester internship experiences. 

Phrase Nets to Enhance Reading Comprehension. 

Undergraduate students were given an assigned reading in an online book chapter followed by an 
assessment of their understanding for 1) main ideas, 2) relationship between terms in the readings, and 3) relevancy 
to readers’ personal learning experiences. The participants were also asked to provide a self-assessed level of 
confidence in their interpretations of content from the book chapter. 

Following the reading assignment and assessment of understanding, students were instructed to access the 
website Many-Eyes.alphaworks.ibm.com and generate a visualization using the Phrase Net tool. Phrase Net 
analyzes a text by looking for pairs of words that fit particular patterns. In Figure 1 is an example from a science 
article on the solar system. The user will specify a particular pattern such as the terms earth or planets. Phrase Net 
will create a network diagram of the words it found as matches. The terms earth/planets will be connected if they 
occurred in the same phrase. When viewing the output from the visualization, the user can compare differences in 
font size, color, and size of connectors to determine frequency and relationship of the terms. The size of a word is 
proportional to the number of times it occurred in a match. 



Your visualization will look like this: 





ancient 



solar 




small 



stars 

magnetic 



Figure 1. Visualization using PhraseNet 

Note in the figure above the tern “dioxide” is connected to terms “atmosphere” and “carbon”. Bold lines indicate a 
strong connection between terms “solar” -> “system”; and “magnetic” -> ”field(s)”. Drawing attention to strong 
and weak connections between ideas enhances the reader’s conceptual understanding of complex reading material. 

Undergraduate students who participated in this study completed visualization activities followed by a post 
assessment to measure reading comprehension and confidence levels when using the Phrase Net tool. Students’ 
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responses were rated 0 to 4 with 4 being the highest score possible. T Test analyses were used to compare pre and 
post assessment for the following: 1) understanding main ideas; 2) comprehension of relationships between main 
ideas in the readings; 3) ability to make connections between main ideas and personal learning experiences, and 4) 
self-reported confidence levels in understanding and comprehension for the readings. Differences in Pre assessment 
and Post assessment of main ideas when using the Phrase Net visualization is marginally significant (p < .04, 2- 
tailed). No significant differences were identified between pre and post assessments relating readings to personal 
experiences. However T Test comparison shows a significant difference in assessment of students’ ability to see 
Relationship Between Ideas when using the Phrase Net tool (p < .000, 2 - tailed). Note in Table 1 differences in T 
scores (T = H-3; p < .000, 2 - tailed) for the pretest [before use of Phrase Net] compared to T scores ( T = 14.3 , p < 
.000, 2 tailed) for the post test score [after use of Phrase Net]. 



Table 1. T Test comparison of students’ ability to see relationships between ideas when using visualization tools. 





N 




Mean 


SD 


Std Error 




RelationPre 


14 




2.5714 


.85163 


.22761 




RelationPost 


14 




3.0000 


.78446 


.20966 














Confidence Interval 


t 




df 


Sig. (2-tailed) 


Mean difference 


lower 


upper 


RelationPre 11.298 




13 


.000* 


2.57143 


2.0797 


3.0631 


RelationPost 14.309 




13 


.000* 


3.00000 


2.5471 


3.4529 



Students’ confidence levels in identifying relationships among ideas also appear to be impacted through the use of 
Phrase Net visualization tool. In Table 2, differences in pre and post self-reported confidence levels is significant 
(p < .000, 2-tailed). 

Table 2. T Test comparison of Pre use of Phrase Net with Post use of Phrase Net when measuring students’ 
confidence in ability to identify relationships of main ideas in assigned reading. 



Confidence in Identifying Relationships Between ideas 





t 


df 


Sig. (2-tailed) 


Mean difference 


Confidence Interval 


ConfRelaPre 


6.852 


13 


.000* 


1.55643 


Lower 

1.0657 


upper 

2.0471 


ConfRelaPost 


8.962 


13 


.000* 


1.95786 


1.4859 


2.4298 



Students’ comments following the use of Phrase Net seem to support the positive effect of the visualization 
tools used to help students understand relationships between major terms appearing in assigned reading. One 
student commented: 

To visually see the connections of how strongly 2 concepts are linked to one another made me view this 
article in a different light. Previously, I selected main subjects points that were not as important or had 
fewer connections compared to the ones that I have selected after using PN. It reinforced the main issues 
that were being discussed in the article, rather than focusing on mini subjects that ewer not he key points. 
For example, PN greatly reinforced the strong connection between synapse over production and loss, which 
is an imperative method in how information is translated into learned concepts. 
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Ideas for Using Word Clouds in K-12 Classrooms. 



Both graduate and undergraduate students provided innovative ideas for use of visualization tools for K-12 
classroom instruction. In response to a discussion forum asking for possible uses of Wordle (Wordle.net), students 
submitted ideas for a classroom activity and method for evaluating outcomes from the lesson. See Table 3 for a 
selected sample of classroom activities designed by students participating in the study. 



Table 3. Instructional activities using word cloud and tag cloud visualization tools. 



Skills 


Instructional Method 


Outcomes 


Assessment 


Identify main 
idea 


View word cloud for each 
paragraph one page narrative; 
compare key words; write one 
sentence with main idea for each 
paragraph. 


Students made connection between font 
size and number of occurrences in the 
narrative; discussion on meaning of 
prominent words. 


Teacher observation; 
enhance concept of 
"Big Idea" 










Critical 

thinking 


Individually generate a list of 
terms associated with educational 
theme; groups generate a 2nd list 
of terms; independently generate 
word cloud from assigned 
reading on the theme; reflect and 
discuss comparison of 
independent and group lists 


Critical thinking required to compare 
and contrast word lists; accurate 
analysis of assigned reading; self- 
assessment associated with reflections 
on personal word clouds representative 
of themes in the reading 


Comparison of word 
clouds representative 
of themes and main 
ideas within the 
reading; knowledge 
and understanding of 
vocabulary in the 
reading. 










Expressive 

Writing 


Assigned writing describing 
yourself; paste into word cloud 
software; compare characteristics 
with others in class; identify 
repeated words; reflect on self- 
image and your best attributes 


Creative writing; good use of adjectives 
and adverbs; positive reinforcement for 
all students; identify the purpose of 
"sparkle words" for narrative writing. 


Teacher assessment 
of writing style, use 
of vocabulary and 
positive dialog during 
whole class 
discussions. 



The researchers summarized all students’ word cloud activities and placed these into categories. Although more than 
30 student assignments were submitted, we synthesized these into three main themes: Reading to Identify Main idea; 
Critical Thinking Skills, and Expressive Writing. Table 3 provides sample activity for each of the three themes. 

Summary/Conclusion 

Use of visualization tools as a pre or post reading activity has potential to enhance reading comprehension 
in adult learners. Although the researchers consider this a preliminary study, pre and post tests reveal a positive 
effect through use of Phrase Net tool for identifying relationships in ideas in expository reading. Feedback from 
participants suggests a positive effect on reading for understanding using complex college level materials. The 
readers were able to make connections between key ideas within the narrative. Use of visualization tools reinforced 
what some readers identified as related terms within the assigned book chapter; for others use of the visualization 
tools resulted in “aah ha” moments with new understanding in the readings. The researchers also suggest that 
visualization tools might aid the generative thinking processes as students recognize conceptual relationships within 
the text. Students are motivated to use visualization tools resulting in cognitive maps. By viewing the visualizations, 
the participants in the study were able to participate in discussions based on a mental map of abstract concepts and 
ideas. We consider differences in students’ pre and post tests strong enough to warrant continued work in this area 
of research. 

Use of word clouds and tag clouds provided imagery representing the many tasks and responsibilities of a 
technology specialist in a K-12 school. As students generated original word clouds and view output from others in 
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their class, ideas for internship proposals showed a strong correspondence with national standards for a professional 
position in K12 schools. Content within the proposals from this group, compared with earlier internship proposals 
from previous semesters, showed an increase in clarity of ideas, consistency with standards, and innovation in 
proposed projects. 

Participants in this study were motivated to use visualization tools in the design of K12 classroom 
activities. Graduate students in instructional technology courses and undergraduates in pre-service teacher classes 
designed classroom activities using Wordle.net. The activities were designed for critical thinking, problem solving, 
and other higher order processes for K-12 students. Based on results of this study we propose visualization tools can 
be useful to enhance digital literacy and improve reading comprehension with young and adult learners. 
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Introduction 

This study focuses on students’ digital voices as seen in their willingness to use facebook and their 
perceptions of Facebook use as the LMS in their Composition II research writing course. Evaluation of students’ 
use as a social tool as well as an academic learning tool was the primary focus of the study; additional information 
emerged from the data regarding the instructor’s role within the Facebook group and Facebook’s impact on face-to- 
face class meetings. 

Students - Learners - in the 21 st Century have a swirl of mobile technologies, software applications and 
virtual realities circling their daily experience as they maneuver through their lives and their coursework. They have 
constant access to cell phones and use them to not only receive and make phone calls, but to send and receive text 
messages and send and receive emails as well. Students also receive constant updates from their social networks on 
their cell phones, if they have programmed their accounts to accommodate such postings. Students also have many 
software applications available to them on their cell phones or their laptop computers. They can create, open or save 
any type of document they have the application for; they can follow news and information feeds for any RSS they 
are subscribed to; they can tweet the latest news, the answers on the test or their emotional status via Twitter to 
everyone who is fortunate enough to be part of their Twitter circle. These students find everything they need online, 
and they truly believe that everything they do need is online. These same students, who are constantly connected 
with their 1500 Facebook friends through two or three technologies, will check in to their academic course websites 
only every couple of days - or right before they go to class. They may login to their email accounts only once a day 
or every couple of days because they “don’t have time” to check email or the course LMS. 

Course websites that are associated with students’ courses may take a variety of forms. Instructors may 
develop their own websites for their courses and manage the curriculum in that fashion. Institutions may provide a 
Learning Management System (LMS) for their faculty to use; those same institutions may also expect online courses 
to operate on the LMS chosen by the school officials. Systems popular among colleges and universities include 
WebCT, Blackboard, D2L, and Moodle, and they are structured to provide a virtual classroom that has programming 
for submitting assignments, having discussions, monitoring grades, reading or watching lectures, listening to 
podcasts or linking to recommended websites - just to name the most common characteristics of an LMS. 

Developed and promoted in the 90s, the LMS still operates largely on an instructor-centered paradigm, and 
the structure of the website has been found to be limiting because it is not open to activities happening outside of its 
realm (Martindale & Dowdy, 2009, p. 6). “The current one size fits all” (Saba, 2008, p. 2) LMS can be limiting in its 
effectiveness as trends in higher education indicate that an industrial type management of education does not benefit 
learners in this post industrial, mass production era (Saba, 2008). Using variables of structure and dialogue, Saba 
(2008) found that “programs with little structure, dialogue and easy interaction between the learner and the teacher 
permits very personal and individual learning and teaching” (p. 3). Using an LMS can easily become a virtual 
correspondence course where communications occur frequently, but only between the instructor and individual 
students. Paavola and Hakkarainen (2005) posit that students should be “prosumers” (both producers and 
consumers) of knowledge, ideas and artifacts. This concept builds on common elements of Bereiter’s (2002) theoy 
of knowledge building and Engestrom’s (1987, 1999) theory of expansive learning. In an academic era where many 
educators seek to develop a course culture of community and exchange, a learning management system may or may 
not promote that type of paradigm shift. 

The concept of Personal Learning Environments has developed among education theorists, practitioners 
and professionals that want to take advantage of online learning environments, but want to do so without the 
boundaries - digital and institutional - of the typical LMS. Personal Learning Environments (PLE) is defined as a 
supportive online learning environment that is developed by and for the leamer/user, but at this point the definition 
is taking two forms. Some theorists would define a PLE as a specific online organizational tool, and others would 
consider a PLE to be a metaphor to explain the multiple and varied activities of 21 st century learners (Martindale & 
Dowdy, 2009). However, until the PLE is more defined and can reach across academic and institutional barriers, a 
Learning Management System is helpful to manage courses, even considering the institutional and LMS limitations. 
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Certain tools are available on the web that resemble what these theorists characterize as a PLE, and many of them 
are open source. There is no need for a software application that reaches across the entire institution, and learners 
can use these tools to create individual environments that accommodate their resources, activities, products and 
interests. PLEX, Colloquia, Elgg and Netvibes are a few website organizers that could function as a PLE; Facebook 
is also included in the list of potential PLE tools (Martindale & Dowdy, 2009). The goal of integrating any of these 
tools into the learning experience would be for students to contribute to the community’s existing body of 
knowledge as they participate in the sociocultural practices of the learning community (Lee et al. 2005). 

Description of the Study 

Since students already used Facebook on a consistent basis, this study explored the idea of using the 
popular social network as an LMS because it has the flexibility of a Personal Learning Environment, but it might 
also provide the benefit of a frequently visited virtual classroom. Students turn to Facebook to communicate with 
friends, to post positive or negative emotions, to notify their Facebook world about an event, to obtain quick 
information or to make arrangements about a class or a meeting or a party. The use of e-mail is minimum compared 
to Facebook. Almost one third (28%) of the students who participated in this study reported spending three or more 
hours on Facebook each day, while 1 1% of students reported spending five plus hours per day. The largest 
percentage (39%) of students said they spent approximately one hour per day on Facebook. 

This study was guided by the following research questions: 

• Flow did students engage with course content using Facebook? 

• Flow did students engage with the instructor using Facebook, texting, and e-mail? 

• Flow did students feel about adding classmates and instructor to their Facebook list of friends for the 

purposes of this course? 

• What do students like and dislike in general about Facebook? 

• Were students positive or negative about the use of Facebook in this class? 

Background 

While preparing for a new semester of courses, the primary researcher learned that the institution she was 
teaching for would not provide access to an LMS that year. The institution was in a transition between adopted LMS 
tools, and they would not have one available for faculty that semester, nor was the issue addressed, encouraged, or 
discouraged. She considered that the students might all have Facebook in common and decided to approach 
engaging them in class and in their writing development by using a social network with which they were probably 
familiar. Most students constantly receive Facebook status updates on their iPhones and cell phones, but would they 
be as receptive to receiving updates and messages from their Comp II class? On the first day of class the primary 
researcher announced her intentions of using Facebook as the LMS for the course and instructed each of them to 
‘friend’ her before the next class. Initial responses from students included surprise that their instructor was on 
Facebook as well as general apprehension that is typical with freshmen and sophomores trying something new in a 
course. 

Professionals, parents, students, adolescents, college kids, friends, co-workers - all types of people - are 
using Facebook. With a total of 500 million users (Facebook Press Room), it is the most popular of social networks 
among others such as MySpace, Linkedln, and Proxy. When narrowing the focus of Facebook users to college 
students, the numbers are amazing; 96.7% of 600 students studied had established a Facebook page (Benson, 2008). 
Seventy percent of Facebook users live outside of the United States (Facebook Press Room), and students spend 
time on their social network spaces each day, and often the time and attention devoted to Facebook exceeds the time 
and attention spent on studies and coursework. 

Setting 



This study was conducted at a private liberal arts university of approximately 4000 undergraduate students. 
The course using Facebook as an LMS was a single section of International Composition II. The primary researcher 
and instructor for the course had never taught a composition course targeting international students. The course 
convened face-to-face three times per week from 12:30 - 1:20 p.m. (CST). The physical classroom featured a 
traditional arrangement of tables and chairs in rows and one classroom computer and projector. 
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Participants 



Twelve undergraduate students were enrolled in the course. Fifty percent of the students were registered as 
sophomores, 38% freshmen, and 13% juniors. The first student in the classroom on the first day of class was a 
Mexican American student whose only connection with her ethnic heritage was her surname. She was born and 
raised in the United States, and English was her first language. The primary researcher noted students’ ethnicity and 
informally evaluated their oral English skills as they entered the classroom. 

“Xie” from China arrived, and he watched every move the instructor made and tried to understand every 
word spoken. He had been in the United States for nine months but had been enrolled in English classes since he 
was six years old - a total of 18 years. He kept his laptop open in order to constantly look up definitions to words he 
heard and words he wanted to say as discussions developed around him. “Betty,” a United Kingdom student, came 
into the classroom, and she sat off to the side seeming to send out a message of isolation and distance. And then 
“Teona” from Jamaica entered the room. She seemed almost agitated and her English was certain, but it was 
definitely Jamaican English. We all had to listen very closely to understand familiar words with an unfamiliar, 
quickly spoken accent. 

The remaining six students in the classroom that first day were a surprise: they were 100% English first 
language, American students. This was contrary to the instructor’s expectations of an “International Comp II” class. 
One was from Rhode Island, one was from California, and the remaining four were from the heartland of America 
within 100 miles of the university that had brought us all together. 

The final two members joined the class about two weeks into the semester. They were both from Saudi 
Arabia, but they did not know each other personally before they had been enrolled in this section of Composition II. 
Arabic was their first language, and both had been studying English in America for over a year and were just in their 
second semester of course work at this institution. “Ma’mar” was more proficient in speaking and writing English, 
and while “Ibkar” was all smiles and braces, he had to listen carefully and respond slowly to communicate in 
English. 

All of the students were computer literate; Xie, Ma’mar, Teona, Jacey, Alan and Cliff all brought their 
laptops to class each day. Betty, a self-admitted Facebook addict, did not actually own a laptop computer but had 
been a member of Facebook for four years. Half of the participants had owned their Facebook for three years, and 
the remaining three participants who responded to the question related that they had been part of Facebook for two 
years or less. 

Methods 



Using social networking tools to help international students engage in face-to-face class activities and to 
improve their writing and learning was an interesting experience. On the first day of class, during the syllabus 
review, the students were instructed to ‘find’ the instructor on Facebook and to ‘friend’ her. The privacy settings 
were adjusted in such a way that the instructor’s profile could not be located with a general search, and when she 
entered the classroom on the second day of class, the students were distraught and asked right away about the profile 
and if they had missed anything since they had been unable to friend the instructor. 

Throughout the 16-week semester, students were directed to use Facebook for various assignments, links 
were posted on Facebook, discussions were posted and group communications were established. The instructor 
created a class group and engaged in Facebook chats and exchanged messages with students. Additionally, the 
instructor built a wiki site that was linked from the class Facebook group page to be used for posting assignments 
and also connected to an open source gradebook through a link on the course Facebook page. In the final weeks of 
the semester the instructor conducted a panel interview and asked students to share their perceptions about their 
current coursework in conjunction with Facebook, as well as their ideas for future uses of Facebook in other course 
work. This panel interview began with structured questions and developed into a wide ranging discussion on 
Facebook issues as well as the influence of Facebook on students’ writing and other scholarly activity. 

Researchers administered a brief survey using Google Documents in order to gather demographic data as 
well as online course experience background and perceptions. 

Results 



The results indicated that most students used Facebook an hour or more per day. Most students - 75% of 
those involved in this study - checked their messages and notices as soon as they logged on to Facebook, or they 
had their messages automatically forwarded to their cell phones. When the instructor had information to get to 
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students, the quickest way to contact them was by a text message to their cell phones. It was almost as quick was to 
send a message to the entire group through the messaging feature in Facebook. Students would instantly reply to 
instructor messages. This strategy was discovered inadvertently by announcing a change in class meeting time. It 
worked so well that the instructor continued to use the messaging feature of Facebook to send out assignment 
reminders. Students engaged more in questioning through Facebook messages directed to the instructor than asking 
them verbally in the face-to-face classroom. 

The instructor used Facebook scores of times throughout the semester to send messages to individual 
students and messages to the entire group. 100% of the time, the group messages regarding assignments or class 
notices received no reply. When students received individual messages - asking about an absence or commenting 
about an accomplishment outside of class or a situation in class - they would reply to those messages. Twice in the 
semester students initiated a message; the rest of the messages were initiated by the instructor to students, either as a 
group or individually. 

Group messages about class or an assignment or due date on the Facebook group page led to a flurry of 
emails and text messages to the instructor regarding specifics. Students would not message the group page with 
specific questions but would email or text me instead. Words like ‘citation’ or ‘evaluation’ would be difficult for 
ESL students to understand, and they seemed to be more confident about understanding the instructor’s answers 
when they had spoken face to face. 

When engaging with the course in Facebook, students reported first checking messages or notifications 
when they logged on. Students were eager to confirm the specific use of Facebook the instructor was expecting. 

Four students were very comfortable adding the instructor and classmates as friends on Facebook. Others were less 
excited, noting that “there are some types of things that you want to share with peers only” and that “Chinese parents 
are manipulative.” One student explained, “I am Facebook friends with a lot of my family members and im starting 
to realize that it's a little bit of an issue, especially with college life. They comment on everything, end up talking to 
my friends on my page and seeing pictures of my personal life all over the place.” 

Students were mixed in their general preferences in using Facebook. Some liked “keeping in touch with 
friends and family” and one student appreciated the opportunity to stay in touch with a few people they had met 
while in Europe for two weeks. Others wished for more privacy, particularly when being “tagged” in pictures. All 
agreed, however, that “Facebook is changing the way people communicate.” Two-thirds of the participants believed 
that Facebook had an effect on their social lives and on their academic achievement. One-third of the participants 
believed that Facebook had no influence on any aspect of their life - they viewed the social network as random 
entertainment. 

The students in this study had relatively little experience with any type of LMS. The truly international 
students had no concept of how classes were managed in an online fashion. The few students who had taken online 
courses had experience with WebCT and D2L. Their experience with Facebook as an LMS was a standalone 
experience rather than a comparison experience. All participants, except one, were interested in using Facebook as 
an LMS because they wanted to be incorporated into American student culture in as many ways as possible. 
Regarding the use of Facebook as an LMS in this class, students were unanimously positive, noting that Facebook 
made “it easier to connect with my teacher and classmates if we have a group project or if we are having trouble 
with homework” and “it makes it easier to communicate with the class.” One student just responded that “it’s fun.” 
while another noted, “I definitely like it because I get my Comp 2 messages sent to my cell phone.” “When I miss I 
am able to find out what I missed” and “in case I forget to write something down we have it on our page so that I 
can check on it anytime to see when something is due” were also offered as positives. Teona, the student who did 
not want to use Facebook in the Comp 2 course also did not want to use Facebook for any other reason. Her 
Facebook friends were only the people from her Comp 2 class - a total of 13 including her and the instructor. When 
asked if Facebook helped or hurt their success in school, participants indicated that Facebook was both good and 
bad. They could quickly communicate with classmates about school issues and school work, but they also could be 
distracted easily once they logged on. One student said, “[It] helps because I keep up with classmates and homework 
on [Facebook], but it hurts because if I have to type a paper, Facebook is always open and much more enticing than 
homework.” 

The distraction factor is significant from students’ perspectives. They receive updates and can message 
others from their cell phones when they are not at a computer. When they begin doing their homework on the 
computer, they are tempted to check on their friends and chat, and they end up procrastinating on homework. 
Without realizing it students spend much more time on Facebook than on their studies. Students may creep on 
pictures for an hour or more before they realize they have not even looked at their homework. Facebook can also be 
a problem when students are in class because they may check their profile page instead of listening and participating 
in class. 
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Students’ feelings toward Facebook being helpful or harmful were mixed. In general, they appreciated 
using the social media tool to connect with friends and family and improve communication, but most students 
mentioned the harm of spending too much social time on Facebook rather than doing homework. Overall, students 
see more help than harm in using Facebook. They appreciate the benefit of staying in touch with friends anywhere 
in the world. Xie used the Chinese version of Facebook for his Chinese friends in America as well as a 
communication tool with friends in China when they were able to get online. As international students, this 
included their friends from their home country and new friends in America. Ma’mar and Ibkar used Facebook as a 
social tool extensively, often posting their party plans and escapades as status updates. They had no worries about 
their female instructor holding them accountable for anything. One particularly candid student explained, “instead of 
researching I just end up playing games or talking to people, but even if there wasn't facebook I would find 
something else to distract me.” 

International students viewed Facebook as a place to express themselves or even to reach out to others. 

One lonely Friday night, Xie spent the entire evening ranting by periodically posting 75 or more status updates that 
reflected his anger. He ranted in English, and some of his Chinese friends replied in Chinese. Xie’s status update 
rant was truly a cry for friendship and understanding. If he were just trying to express himself, he could have done 
so in Chinese. 

Facebook is like a vehicle, money, or even food. It can be used unwisely or it can be a powerful part of a 
person’s life - in the classroom, in online cultures or personally. Instructors can use Facebook as a way to engage 
students in course work. If reluctant students, or International students, or even underprepared students can establish 
a point of connection with a group of classmates or an instructor, then their willingness and ability to commit to the 
coursework increases tremendously. In the group of students that participated in this study, not one student dropped 
the course. Finishing with 100% of the students that begin a course is a rare experience in undergraduate classrooms. 

Overall, the study successfully used Facebook as an LMS in and International Composition II course as 
determined by answers to the research questions. A final point for consideration is the fact that annually hundreds of 
thousands of dollars are spent on university hosted course management systems while open source social networking 
systems are available and provide similar or even superior features. University implementation and support of a 
proprietary LMS costs a large subscription fee and sometimes an accompanying support service fee. Some open 
source LMS models exist, but the support for those systems may also have to be purchased or training may need to 
occur for (paid) on-campus personnel to support questions and problems that arise throughout a course that is using 
the LMS. The individuality and personal attention that many students and instructors involved in education often 
prefer may actually be more attainable in an online environment if social networks like Facebook are used to 
promote communications. The often sterile, person-less atmosphere of an online course using a traditional LMS 
will be far less likely to engage students with their instructors or with their classmates than the social networks like 
Facebook where students spend large amounts of time and energy. 

Future studies on this topic should include a comparison of two similar courses - one that uses a traditional 
LMS and another that uses Facebook as an LMS. Also, instructors should consider using a few of the 550,000 
applications available on Facebook to increase the opportunities of students in a Facebook group to operate within 
that group and within that Facebook page. 
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Background 

Web 2.0 technologies are making an impact in the communicative behaviors of individuals. Abbitt (2007, p. 1) 
stated that there has been “tremendous growth in the popularity of websites focusing on social activities and 
collaboration”; this would include online applications such as Facebook. Facebook describes itself as a “social 
utility that connects people with friends and others who work, study and live around them” (Facebook, 2008, p. 1). 
Flowever, the research related to what kind of interaction could occur in Facebook and be beneficial to students’ 
learning is still limited in Taiwan. The study focuses on bridging the gap based on research that examined students’ 
perceptions about what kind of interaction could benefit their learning in a class integrating Facebook. 

Interaction is considered to be a necessary and important ingredient for a successful learning experience so 
distance learning practitioners and researchers have concerned with how much interaction a distance learning 
environment could provide for students (Mclsaac & Gunawardena, 1996). 

Activity theory was applied in analyzing students’ interactions in a Web 2.0 tool— Facebook environment. 
Activity theory (Leont’ev, 1978; Vygotsky, 1978) is increasingly being used to explain social aspects of technology- 
supported learning (Jonassen, 2002). Activity theory can contribute to computer-supported collaborative learning by 
“[understanding] learning not as the internalization of discrete information or skills by individuals, but rather as 
expanding involvement over time — social as well as intellectual — with other people and the tools available in their 
culture” (Russell, 2002, p. 65). 

There are seven elements in the model and the definition of each element as follows (Jonassen, 2002; Jonassen 
& Rohrer-Murphy, 1999; Collis & Margaryan, 2004): 

1. Subject: The individuals such as leaner, peers, facilitator, supervisor, instructor, and mentor who 
participated in the activity. 

2. Instillments: Methods, resources, supports, online tools and environments that facilitate the activity such as 
technological tools, networks, and learning resources. 

3. Object: Products created by the subjects during the activity such as learning tasks, assignments, and 
projects. 

4. Community: Socio-cultural environment in which interactivity takes place such as virtual classroom and 
organization. 

5. Rules: Standards and norms of the community that rule the activity such as frameworks, culture and other 
standards that influence the learning environment. 

6. Division of Labor: Roles and relationships within the community that affect task division and 
responsibilities. For example, roles and relationships within cooperative teams and courses. 

7. Outcome: The overall results achieved by the activity system such as final products and learning outcome. 

The purpose of this study was to find out students’ attitude toward their interactions with peers, instructors, 
course material, and interface in a class integrating Facebook. The study tried to find out the kinds of interaction 
facilitating students’ learning, the barriers encountered, and suggestions to overcome these barriers were 
investigated in the study. 



Methodology 

A purposeful sampling was used to select participants in a university in northern Taiwan in 2010. An intact 
class of 32 participants was used in this study. The participants were asked to use Facebook to take part in the 
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discussions for each topic. The treatment lasted for 8 weeks. After the treatment, thirteen participants voluntarily 
took part in the in-depth interviews to provide an in-depth portrait of their experience of using Facebook in terms of 
interacting with others. The interview was semi-structured and questions were related to students’ interaction 
experiences in Facebook. In order to completely understanding the feelings of the participants, the qualitative 
research was conducted for researcher to go deep into the interviews. The whole interview was video-taped. The 
video-tape was then transcribed by the researcher and a Grounded Theory Analysis based on Activity theory was 
used to analyze the transcripts. 



Results and discussion 

The transcripts were organized according to Activity Theory and the results provided the following findings: 

Subject — Instrument Interactions: Facebook was easy to use 

Most learners agreed that Facebook was simple and convenient to use. Most learners were familiar with the 
Facebook environment because they already used Facebook in their daily life. Most students didn’t need to learn 
how to log in, get a new account, or post a message since they already did so before the class. 

“I use Facebook almost every day so I’m familiar with most functions provided in the 
Facebook.” 

“It’s convenient to me because using Facebook is part of my life so I don’t need to use another 
website to discuss with my classmates.” 

Most students mentioned that wall post enabled teachers to announce news and remind learners when the 
weekly discussion was due. Besides, Online Chat enabled synchronous discussions among teacher and classmates. 
Some students used Online Chat to ask teacher and classmates questions. 

“I usually use Online Chat to ask questions about assignments. The response is immediate 
and I won’t feel embarrassed. I can also check wall post to get the latest news about the 
class.” 

Subject — Object Interactions: Learn others’ ideas by observing them 

Most students agreed that they learned others’ ideas and experiences from reading classmates’ messages. 
Learning from others was regarded to be helpful to them. 

“After reading others’ postings, I am usually stimulated from other viewpoints and I think 
this is helpful to my learning. I can also know my classmates more.” 

Photos and videos of students’ presentations were taken in the class. Most students mentioned that sharing class 
photos and videos in Facebook motivated learners to participate in Facebook and learned from others’ presentation. 

“I will watch the pictures and videos taken from the class. I think those are interesting. I also 
observe how my classmates present in the class.” 

Subject — Community Interactions: Students felt comfortable to express themselves in discussions 

Most students felt comfortable in the Facebook environment and felt comfortable to express their ideas. Some 
students may not be willing to share their experiences or ideas in the class but Facebook provided the environment 
for students to discuss and ask questions. 

“I feel comfortable to express myself and I don’t like to speak up in the class. It is too much 
pressure to express my ideas in the class.” 
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Outcome — Understanding the course materials and interaction Increased 

Most students expressed that when using Facebook to take part in the discussions, they studied their textbooks 
and searched the Internet to find out answers. They also asked teacher and classmates questions to solve their 
questions. Their interactions increased in the discussions and through Online Chat. 

“In order to participate in the discussions in Facebook, I studied and understood the course 
materials more. I also asked teacher and classmates questions to complete my assignments.” 

From the interview, the participants shared the problems when using Facebook and some recommendations 
provided by the students: 

Subject — Instmment Interactions: Editing functions 

Most students expressed that the editing functions in Facebook was still too basic and more font styles as well 
as colors should be provided. 

“I always use Microsoft Word to edit my answers first and paste the content to Facebook. The 
Facebook should provide more functions to make editing more easily.” 

Subject — Instrament Interactions: Uploading different types of files 

Most students recommended that providing course materials would encourage students participate more often 
in the Facebook as well as become more familiar to the course contents. However, only pictures and video files can 
be uploaded in the Facebook so far. Enabling more different types of files such as doc, ppt, and pdf files to be 
uploaded in the Facebook was recommended. 

“I think if there are more course materials in the Facebook, I will be more willing to use the 
Facebook and it will be more convenient for me to read the materials and take part in the 
discussions.” 

Subject — Instmment Interactions: Learning management 

Some students felt that record learners’ number of postings and scores would be helpful to them. A reminder 
when they haven’t completed certain discussions was also recommended. 

“Sometimes I don’t remember which discussions I haven’t completed. I need a summary to 
let me know my learning portfolio and remind me if I haven’t completed.” 

Subject — Instillment Interactions: Group sorting function 

Some students mentioned that providing the group sorting function would help them find the learning group 
used in the class more easily. Some of the students have participated in many groups already in the Facebook. 

“I have many groups in the Facebook already. It will be convenient for me if I could sort the 
groups that I participate in so I don’t need to spend time searching for the group used in the 
class.” 



Conclusion 

From the results presented above, it is concluded that most students felt the Facebook was easy and convenient 
for them to use as a discussion tool in the class. The wall post function enabled them to know the latest events 
happened in the class and reminded them what assignment was going to be due. However, most students mentioned 
that the editing function in the Facebook was still insufficient so they need to edit in other software such as 
Microsoft Word and then paste the content to the Facebook. Moreover, only pictures and video files were able to be 
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uploaded to the Facebook so enables more types of fdes such as doc, ppt and pdf fdes will encourage students to 
read course contents and use Facebook more often. 

Most students mentioned that using Facebook was part of their life so majority of them were familiar with the 
functions and they felt comfortable to express themselves in the Facebook. It was convenient for them because they 
didn’t need to apply for another set of user name and password. Most learners agreed that they learned from reading 
others’ postings and share their ideas help them to understand the course topic more. 

The Online Chat function also helped students to ask questions and communicate with others for immediate 
feedback. Watching class pictures and videos also motivated them to use Facebook and took part in the discussions. 
However, there are some recommendations such as group sorting functions for those had many groups already and 
learning management function for students to create learning portfolio. 

Although Facebook is known to provide a social networking environment, instructional designers and 
instructors should be aware of students’ interaction experience in order to provide a better learning environment in 
the Facebook and help improve students’ learning outcomes and satisfactions. 

Recommendations 

For further research, eight weeks of treatment may be too short for students to interact with each other. A 
longer period of time is recommended. The treatment could be conducted in a fully web-based class to find out the 
interaction types that may differ from this study. A bigger sample size is also recommended to see if other kinds of 
interaction become essential in the Facebook environment. 
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Introduction 

Educators in higher education have started to integrate Web2.0 technologies for enhancing learning and 
instruction activities, taking advantages of their accessibility, ease of use, and potential in achieving various types of 
learning goals. Furthermore, many of the Web2.0 technologies have been designed and promoted as social tools 
which are conducive to interaction, communication, and collaboration (Gunawardena, Hermans, Sanchez, 

Richmond, Bohley & Tuttle, 2009)— the important skills to develop for the 21st century learners (Lesgold, 2009). 
Blog, the shorthand of weblog, is one of the Web2.0 technologies that have captured educators’ attention in higher 
education for its potential for cyberlearning. 

Technologically, blog systems offer an intuitive platform for publishing and commenting on the web with a 
content delivering mechanism that automatically updates the bloggers (i.e., blog authors) and readers about the latest 
content (Kim, 2008). This intuitive platform opens up the opportunities for contribution and participation on the 
Web in this information age (Hsu, Ching, & Grabowski, 2009). Pedagogically, blogging activities can help students 
develop a range of essential skills, such as writing, reflection, and critical thinking skills, depending on the desired 
learning goals. From the perspective of learning, “knowledge is created, shared, remixed, repurposed, and passed 
along” (Mason & Rennie, 2008, p. 10) in a Web 2.0 environment such blogs. Socially, blogs create virtual spaces for 
interaction between the instructor and students, among the students, and between students and the public with access 
to the web (Fessakis, Tatsis, & Dimitracopoulou, 2008). With this increased interactivity, blogs enable development 
of communication and collaboration skills and serve as a powerful channel for social networking that encourages 
students to establish their support groups for learning (e.g., learning communities) and/or for professional 
development (e.g., communities of practice) (Gloff, 2005). 

Although blogs theoretically may seem to be a promising technology for learning and instruction in higher 
education, research findings on the educational blogging are not always positive in terms of whether blogging 
activities had helped students achieved the intended learning goals (Divitini, Haugalokken, & Morken, 2005). 
Moreover, the emerging research on blogging at various higher educational contexts reveals challenges and issues 
associated with designing and implementing such activities. The accumulative research findings of successful and 
unsuccessful blogging activities suggested valuable implications for future design and implementation of such 
activities. This paper systematically examined empirical research studies on educational blogging in the context of 
higher education. The purpose of this presentation is to provide considerations for designing and implementing 
effective educational blogging activities to help achieve intended learning goals. Specifically, the following research 
questions were asked: 

1 . What are the learning goals that can be achieved via educational blogging? 

2. What are the issues and challenges of educational blogging activities? 

3. What skills do students need to develop in order to blog effectively for learning? 

Research Method 

This study systematically examined more than 10 existing empirical blogging research studies published in 
peer-reviewed journals to answer the aforementioned research questions. In general, the included studies were 
published between 2003 and 2009. The research articles were selected from these years due to the increasing 
popularity of blogging starting at 2003. Most of the studies were evaluation research or conducted using case study 
method. Table 1 presents the list of the included research articles along with the participant levels, number of 
participants, subject matter, and research method. 
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Table 1. The articles of blogging research reviewed and included in this paper 





Participants’ 
Education Level 


Number of 
Participants 


Subject Matter 


Learning Goals 


Ellison & Wu 
(2008) 


College students 


52 


Not specified 


Writing and peer 
commenting to help 
understand course 
content 


Farmer, Yue, & 
Brooks (2008) 


Undergraduate 

freshmen 


225 


Art 


Student interaction 
through reflecting 
and discussing on 
course content 


Fessakis, Tatsis, & 
Dimitracopoulou 
(2008) 


University 

students 


7 


ICT 

applications 
and product 
development 
for math 
instruction 


Design and 
development of 
simple educational 
applications and 
learning activities 
for technology 
enhanced learning 
environment 


Glass & Spiegelman 
(2007) 


Community 
college students 


Students from 
4 courses 


Math and 
Computer 
Sciences 


Information literacy 
skills 


Ladyshewsky & 
Gardner (2008) 


Undergraduate 

students 


38 students in 
groups of 4 or 
5 


Physiotherapy 
clinical field 
work 


Reflective practice 


Philip & Nichollas 
(2009) 


Undergraduate 

students 


21 


Playbuilding 
in drama 


Analytically and 
evaluatively 
reflective thinking 
skills 


Sharma & Xie 
(2008) 


Graduate students 


8 


Instructional 

Design 


Reflection 


Shoffner (2009) 


Graduate students 

(pre-service 

teachers) 


9 


Not tied in 
with any 
courses. 


Reflective practice 


Tekinarslan (2008) 


University 

students 


42 


Computer 

Usage 


Research and 
writing skills; blog 
usage skills 


Wassell & Crouch 
(2008) 


Undergraduate 

students 


24 


Multicultural 

Education 


Understanding of 
Diversity 


Williams & Jacob 
(2004) 


Graduate students 
in MBA program 


51 students 
from two 
courses 


Macro 

Economics 

and 

International 

Political 

Economy 


Cross-course 

discussion 


Xie, Ke, & Sharma 
(2008) 


Undergraduate 

freshmen 


44 

participated 
but data were 
collected only 
from 27 
students 


Political 
science course 


Reflective thinking 
skills 
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Findings 



Learning Goals 

The analysis indicates that blogging activities were designed to achieve a wide range of learning goals. 

Reflective thinking/practices. Most frequently, blogging activities were adopted to replace the traditional 
journaling activities to promote reflective thinking. Serving as learning journals, blogs provide space for individuals 
to express their observations and perspectives, and to make connections between what they learn and their 
experiences (Gunawardena, et al., 2009). Studies showed that college and master’s students improved their 
reflective thinking through regularly keeping a journal using blogs to reflect on their learning, projects, or clinical 
field experiences (Sharma & Xie, 2008; Xie, et ah, 2008; Philip & Nichollas, 2009; Ladyshewsky & Gardner, 2008; 
Shoffner, 2009). In addition to the reflective skills, students in several studies indicated that they valued 
observational learning because they are able to leam from peers’ experiences by reading their reflection on the 
blogs. The vicarious learning may augment students’ applicable knowledge by assimilating relevant experiences 
from peers. It can also aid on the affective aspect of learning. When students understand that they are not alone 
when facing certain learning challenges, they may reduce their self-doubt and become more confident in their 
learning. 



Critical thinking and writing skills. In another study, college students in the area of mathematics and 
computer sciences developed their information literacy skills such as searching and evaluating needed resources for 
composing while writing blog entries (Glass & Spiegelman, 2007). Preservice teachers in a course of multicultural 
education had applied learned theories to analyze related events and/or news through blogging (Wassell & Crouch, 
2008). By reading peers’ blogs, these preservice teachers also gained critical perspectives to understand the cultural 
issues that they did not have a chance to experience by themselves. Elisson and Wu (2008) explored student 
perceptions of blogging and whether writing and commenting on peer blogs helps students understand the course 
content. They found that students deemed reading others’ blogs helped their understanding of the course content the 
most. On the other hand, making commenting on other blogs or receiving comments was found to be less helpful. In 
addition, commenting activity caused uncomfortable feelings among students. 

Collaboration and communication. Other studies also explored how students took advantage of the 
commenting function and the role of group blogs in supporting group interaction and activities. For example, 
Fessakis and his colleagues (2008) created a group blog for a group of undergraduate students in math education to 
enhance a collaborative “learning by design” activity, with the goal to publish a common set of activities as 
participants’ collaborative product. In Philip and Nichollas (2009), group blogs were also used to foster group 
communication and to serve as a project management platform for a collaborative playbuilding activity in a drama 
course. When working collaborative ly on a project where communication is key to ensure a successful experience, 
research findings suggested that group blogs provide an alternative and valuable space for students to 
communication and collaboration. 



Issues, Challenges and Implications 

While the empirical evidence supports the educational value of blogging activities, issues and challenges 
associated with designing and implementing successful blogging activities also emerged. First, the findings of the 
examined studies suggest that well-structured learning activities are essential for students to perform their learning 
tasks. Among the studies we reviewed, several of them did not inform students about the requirement for frequency, 
length, content, and assessment criteria for their blogging, which led to student confusion or even frustration about 
the learning task (e.g., Sharma & Xie, 2008). This finding suggests that course instructors and designers need to 
provide explicit guidelines on the activity requirement. It is essential to specify the frequency, length, and 
assessment criteria prior to the blogging activity. It would also be beneficial to students by making the guiding 
questions available to prompt their reflect on their practices. For students who are new to a subject matter, these 
questions can be designed to elicit deeper understanding or analysis that exemplifies expert thinking and reflection. 
In addition, to increase students’ motivation in participating blogging activities, reasonable weight in the course 
grade and sufficient time for completion has to be in place. Instructors also need to make it clear regarding the 
purpose (Kerawalla, Minocha, Kirkup, Conole, 2009) and rationale of including the activities so that students do not 
perceive the activity as just another piece of busy work. 
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Second, students often brought into the class their pre-existing perception of blogs and “proper” ways of 
writing blogs (Ellision & Wu, 2008; Kerawalla, et al, 2008). This perception might not be aligned with instructors’ 
vision of educational blogging, which could lead to confusion and low motivation on participating in blogging 
activities. 

Depending on the learning goals, instructors may need to provide guidance for students on reconciling their 
perception of blogging with academic writing. If blogging is used as a way to keep reflective or learning journals, 
instructors may emphasize the non-formal voice that conveys a conversation regarding one’s learning progress 
between students and the instructor. On the other hand, if blogging is used to incubate writing ideas and compose 
drafts toward academic papers, instructors need to make the expectations explicit to students that the appropriate 
tone and writing style needs to be adopted. 

Last but not least, privacy and security issues surfaced in almost all the examined studies. As a publishing 
platform, blogs make it easy for individuals to develop their voices on the Web. However, for individuals who are 
cautious and concerning about the privacy, protecting privacy while publishing on the Web is an issue. This may 
limit their perceived freedom of expression (Divitini, et al., 2005). On the other hand, students who are not aware of 
the privacy issues may make public too much personal information or may violate the confidentiality of others by 
accidentally publishing peers’ information when blogging due to their unawareness. Revealing personal information 
on the Web may lead to security issues. To address the privacy and security issues, many blog systems can be set up 
in a way to protect privacy by making blogs available only to designated group members (students in a course), or 
by making blogs/blog entries unsearchable through search engines. Students can also choose to use pseudonym and 
create a pseudo identity on the Web (Ellison & Wu, 2008). Nonetheless, students would benefit if instructors make 
the issue of privacy and security explicit and discuss good practice of protecting confidentiality with their students. 

Skills for Participating in Blogging Activities 

To participate effectively in blogging activities, students need much more instructional support in addition 
to a technological tutorial on how to navigate through the blogging systems. Our review found that blogging 
activities were designed to help learners in the higher education to develop important skills, such as reflective 
thinking, writing, information literacy skills, and communication and collaboration skills. Many of the reviewed 
studies incorporated a tutorial on the basic functions of blogging systems to train students on using the technology. 
These studies indicated that students usually are able to learn the basic mechanisms of authoring and commenting on 
blogs from a less than one hour of training (e.g., Tekinarslan, 2008). Yet, students were usually inadequately 
prepared for the learning tasks and wished for more instructional guidance on, for example, writing properly for 
specific purpose (Tekinarslan, 2008) or reflecting on learning and/or field experiences (Xie, et al, 2008; Farmer, 

Yue, & Brooks, 2008). When the blogging activities asked students to comment or provide feedback on others’ 
writing, it was found that undergraduate students were ill-prepared for confronting such task due to lacking 
experiences or skills. Students typically did not feel comfortable critiquing, or failed to discuss or comment on 
peers’ work in depth (e.g., Xie, et al, 2008; Wassell & Crouch, 2008; Farmer, et al., 2008; Fessakis et al., 2008). 
Thus, their feedback to peers’ work may only scratch the surface or focus on the technical aspects of the writing. 
Commenting skills involve critical thinking and questioning skills. Students will need to develop these skills before 
they are capable to provide constructive comments comfortably. They would benefit from instructional support on 
how to provide constructive feedback for their peers. 



Recommendations for Instructional Design 

The findings of this review indicated that many studies used blogging activity as an individual activity, 
meaning that no interaction or collaboration was required as part of the activity. While some instructors may request 
or expect that students voluntarily engage in the reading and commenting on each other’s blog entries, the results of 
this review suggests that the interaction between students may not happen without guidance. In their article, 
Gunawardena and her colleagues (2009) compared the focus of Web 1.0 and Web 2.0 technologies and the learning 
paradigms accompanied by these technologies. They distinguished Learning 2.0 from Learning 1.0, suggesting that 
Learning 2.0 emphasizing more on collaborative learning and peer-to-peer interaction, which are fundamental value 
promoted by Web 2.0 technologies. To promote Learning 2.0, instructional design of the blogging activities will 
need to build in mechanisms for interaction or collaboration. Instructional designers may consider incorporating 
instructional strategies that promote pair or group work when designing blogging activities. 
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To promote interaction between students and to develop critical thinking skills, a peer review mechanism 
can be built into the blogging activity. Engaging in peer review activities, students are encouraged to develop skills 
in critical examination and exchange, as well as consider multiple perspectives (Boud & Falchikov, 2007), and 
identify the aspects that had been overlooked. Students are also exposed to opportunities of developing academic 
discourse and reflection of their own learning (Lavy & Yadin, 2010). Research studies also found that the feedback 
provided by peers was taken seriously by students and helped refine students’ original thinking (Lavy & Yadin, 
2010). To guide students in peer review activities through blogging, guidelines on how to interact and how to 
provide constructive feedback need to be in place. For example, the length, frequency, and time for providing 
feedback should be specified. As far as the content of the feedback, it would be helpful to provide questions that 
direct student attention to key areas for constructing feedback. Depending on how familiar students are with the peer 
feedback activity, instruction may be needed to deliberately develop students’ peer review skills and assess formally 
on such skills to increase motivation in skill acquisition (Macdonald, 2003). To close the learning loop, a reflection 
activity will offer an opportunity for students to examine their learning as reviewers and as review receivers. This 
would allow students to be more metacognitively aware of their roles and how each role contributes to their 
understanding of the subject and/or development of other important skills such as critical thinking skills. 

Designing authentic tasks for blogging activity is likely to promote Learning 2.0. This review found that far 
too many existing research studies used blogs as a publishing platform instead of a space conducive for interaction, 
communication, collaboration and critical thinking. To elicit desired interaction and collaboration among students, 
blogging activities need to be designed to incorporate authentic tasks that involve collaboration. Authentic tasks 
have the following characteristics: ill-defined, real-world relevance, opportunities to collaborate, a sustained period 
of time for investigation, and opportunities to detect relevant vs. irrelevant information (Flerrington & Oliver, 2000). 
When engaging student learning in such an authentic task, blogs can be incorporated as collaboration and 
communication tools. For example, in Philip and Nichollas (2009) study, students collaboratively created a play 
over a period of time and performed the play to real audience. During the process, group blogs were used effectively 
for coordination, communication, and management purposes, as there was a real need to interact and communicate 
among the members toward a common goal. As a result, the group blog was used effectively and desired learning 
goals were achieved. On the other hand, when the blogging activity does not involve interaction or collaboration 
with peers, it is unlikely that students will work with each other and view each other’s blogs. 
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Abstract 

The edu-techKNOWiki project enlists the collaboration of the educational technology professional community 
nationwide to develop a knowledge management wiki that organizes information related to educational technology, 
includes a sociogram of professionals in the field, and provides access to an environment of interactive learning 
communities. The interface to the system is based on an analysis of tasks performed in schools using educational 
technology and the socio-cultural patterns surrounding problem solving on those tasks. This innovative system is an 
open educational resource for teacher professional development. 

Introduction 

Information in the area of educational technology is extensive and evolves at such a rapid pace that 
educators struggle to keep up-to-date. Research in teacher professional development related to technology infusion 
demonstrates that short-term strategies such as workshops are not effective. Availability of just-in-time information 
and person-to-person support may be helpful. The emergence of Web 2.0 collaborative technologies marks a 
paradigm shift in the ways in which the Internet is used in schools, away from a depository of content information 
and toward means of collaboration for knowledge sharing and building. This article shares a project in which 
faculty members in educational technology are enlisting the collaboration of the educational technology professional 
community nationwide to develop edu-techKNOWiki, a knowledge management wiki that organizes the knowledge 
base related to educational technology, a sociogram of professionals in the field, and an environment of interactive 
learning communities. 

A wiki is an online collaborative tool. The development of the content of the wiki is by definition a 
collaborative enterprise of the community that uses it. Registered members share creation and editing capabilities. 
Wiki is a word in Hawaiian that means fast; its usage in the lexicon of electronics applies because an electronic wiki 
enables rapid development of content material, based on a “What I Know Is. . .” contribution. Wikis are based on an 
open software architecture system that encourages collective development. Such a system in professional 
development would benefit everyone who used it either to access or to contribute to the development of its 
knowledge system. 

The name edu-techKNOWiki was selected to reflect the centrality of learning to the field of educational 
technology with the embedding of the word KNOW. KNOW also highlights that the wiki is a knowledge 
management system. The beginning syllables, “edu-tech” connect with both educational technology and with the 
Internet suffix .edu since the system is Internet-based. 

This innovative system is an open educational resource for educational technologists, one sector of which is 
teacher professional development. As a professional development strategy it provides teachers who want to infuse 
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technology in their classrooms the opportunity to access constantly updated information as needed and to interact 
with other teachers and educational technology professionals for problem solving. 




edu-techKNOWiki 



Knowledge Base 

The knowledge base contains the substance of the knowledge in the field of educational technology 
including historical, theoretical, and technical information, organized into nodes. It reflects what we have done, 
what we study, and how we behave ethically and think innovatively - about hardware, software and process. 

Conceptual Description 

The field of educational technology is engaged in an iterative process of defining itself. Januszewski and 
Persichitte track the evolution of the definition as part of their commentary with the 2008 definition (2008). The 
first Association for Educational Communications and Technology (AECT) sponsored definition was developed in 
1963 as a definition of audiovisual communications: “Audiovisual communications is that branch of educational 
theory and practice concerned with the design and use of messages which control the learning process. . . the 
planning, production, selection, management and utilization of both 
components and entire instructional systems” (Januszewski & 

Persichitte, p. 260). Subsequent definitions appeared in 1972, with the 
organization’s change from the Department of Audiovisual Instruction 
to the Association for Educational Communications and Technology; 

1977, which formally defined educational technology as a process; and 
1994: “Instructional technology is the theory and practice of design, 
development, utilization, management and evaluation of processes and 
resources for learning” (Seels & Richey, 1994, p. 1). The most recent 
definition was published in 2008. “Educational technology is the study 
and ethical practice of facilitating learning and improving performance 
by creating, using, and managing appropriate technological processes 
and resources.” (Januszewski & Molenda, 2008, p. 1) These definitions document how we think about our field and 
how we have identified the field of educational technology across time. They constitute our reflective knowledge of 
ourselves and of our practice. As such they require a prominent place in edu-techKNOWiki. 

In addition, because educational technology is practiced across a range of contexts including public P-12 
education, higher education, corporate business and industry, health education and Military training, standards and 
elements have emerged from other professional organizations such as the International Society for Technology in 
Teacher Education (ISTE). Their National Educational Technology Standards for Teachers 2008 (NETS-T), for 
example, influence how educational technology practitioners in schools view the field. Teachers think about their 
role with educational technology as reflected in the ISTE’s standards: 

• Facilitate and Inspire Student Learning and Creativity 

• Design and Develop Digital-Age Learning Experiences and Assessments 

• Model Digital-Age Work and Learning 

• Promote and Model Digital Citizenship and Responsibility 

• Engage in Professional Growth and Leadership 

The International Board of Standards for Training Performance and Instruction (IBSTPI) has published 
competencies for educational technologists (instructors, instructional designers, and training managers) across 
training environments. These standards and competencies reflect perspectives on educational technology from 
contexts of practice. 



2008 AECT Definition 




Source: Januszewski & Molenda, 2008 
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As reflected in the names of educational technology academic programs, there are varying perspectives on 
what the field should be called: educational technology, instructional technology, instructional systems technology, 
learning technologies, learning sciences. There are those of us who think the debate over a name has little meaning 
while others consider it of paramount importance to our identity. 

These definitions and standards of educational technology serve as the interlinked structural foundation for 
the knowledge base, but edu-techKNOWiki is more than a collection of published information nodes. As is typical 
with a wiki, each node has an associated page for discussion. The openness and flexibility of the wiki format 
promote transparency and inclusion, permitting all entries, while the discussion screens provide a location for debate 
and discussion. The overall impact of information and discussion contributes to the conceptual evolution of the 
knowledge base of the field. Varying perspectives can be expressed; either consensus can be achieved or alternative 
perspectives can be developed. 

Development process 

The development of edu-techKNOWiki has involved enlisting the collaboration of the community of 
educational technologists. All the research literature on persuasion speaks of the importance of face-to-face contact, 
so over the past year we have visited campuses with large graduate programs in educational technology including 
Florida State University, University of Georgia and Indiana University. During these visits we met with recognized 
leaders in the field and engaged them in thinking about the wiki; two of them were on the definitions committee for 
the 2008 definition. We asked their for input concerning the top programs in the country and names of contacts for 
those programs. We also identified contacts and secured sponsorship of AECT, whose definitions of the field form 
the infrastructure for the knowledge base, and began conversations with various other professional organizations 
including ISTE for similar support. 

To publicize the project and garner support, we presented at regional, national, and international 
conferences: the Eastern Educational Research Association conference in Savannah, Georgia; The Michigan 
Computer Users in Learning (MACUL) conference in Grand Rapids, Michigan; and the Society for Information 
Technologies and Teacher Education (SITE) in San Diego, California. We also wrote a chapter about the project for 
the Educational Media and Technology Yearbook, to be released in February, 2011. 

Annually the Professors in Instructional Design and Technology (PIDT) retreat engages faculty members 
and graduate students from educational technology programs in dialog about issues and new ideas. We attended 
PIDT in Estes Park, Colorado, to get further feedback about developing ideas for the project and to seek 
contributions to the knowledge base. 

To get things started, we have invited contributors to develop content in chunks related to their expertise 
and to link that information to descriptor terms from the 2008 AECT definition of educational technology: 
technology, study, ethics, practice, facilitating, learning, improving, performance, creating, using, managing, 
processes and resources. Category descriptions of these terms are being developed as well. 

The unit of information in the wiki is a node. To establish some continuity and consistency across nodes, 
the example node on planning for technology (by Barry Fishman, University of Michigan) demonstrates suggested 
sections: Roots, Buds, Seeds, FAQs and references. Roots document the historical background related to the node 
while Buds discuss how the topic might influence or be influenced by emerging processes and technologies. The 
format for the node asks for reference to people (leaders, scholars and practitioners in the field) that have importance 
related to that topic (those who have planted Seeds for the developing thoughts). For ease of access to the 
information, each node contains frequently asked questions (FAQs), and references help to give authority to the 
information. 

There are many models through which content for the wiki might be developed. Individual construction of 
a node or set of related nodes is of course open to anyone who creates an account. We urge faculty members in the 
field to create a course assignment in a graduate course to familiarize their students with edu-techKNOWiki as a 
future resource, and to generate content. Based on contacts made at PIDT, we have begun to establish relationships 
to provide a venue for promoting the work of various AECT groups such as the Definitions and Terminology 
Committee and the Publications Committee. The usefulness of this resource for the field and everyone involved in 
the field will depend upon the engagement and contributions of the professional educational technology community. 
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Sociogram 



If organized information in a knowledge base is important to a field, its people share that level of 
importance. Our sociogram is a profile-based representation of the links of educational technologists to the field and 
each other. 

Members of the field of educational technology range from novice to expert. One element of successful 
professional development across the field is mentoring; interaction fosters the evolution of ideas. Faculty members 
mentor doctoral and master’s students. Technology facilitators mentor teachers. Researchers who develop new 
concepts challenge and extend each other’s thinking. While we visited Indiana University, Charles Reigeluth told us 
that as he was focusing on developing instructional theory, interactions with M. David Merrill about similarities in 
elements across theories led him on a new path for one of the later volumes of the “green books.” While some of 
these types of interactions are private and copyright protects intellectual property, a venue for open, transparent 
dialog may be of value both to participants and the field. 

Because educational technology as a profession has developed largely from academic departments, the 
starting point for the sociogram that provides access through edu-techKNOWiki to people in the field is through the 
major academic programs around the country, starting with the following: Arizona State University, Boise State 
University, Brigham Young University, Florida State University, Indiana University, Penn State, Purdue University, 
San Diego State University, Syracuse University, University of Colorado - Denver, University of Georgia, 
University of Missouri, University of Memphis, University of Northern Colorado, University of South Alabama, 
Utah State University, Virginia Tech, Wayne State University. 

Access to different functions in the wiki is restricted by account, although the restrictions are few. 
Anonymous visitors can view anything in the wiki. Individuals who register and have an affiliation with any 
academic or professional role in the field can view everything, create and edit knowledge base pages, participate in 
discussions, make their own professional profile page and use their profile discussion page as a professional blog. 

Some content is protected from almost everyone. For example, a cohort of leaders in the field were invited 
to be project Keynoters. While their contributions are linked into the knowledge base, their Keynoter pages are not 
editable. 

The first entries into the sociogram are these “Keynoters” - leaders in the field who have provided some 
support to the project through their agreement to be featured and honored for their contributions. As an example of a 
graduate student project which both fulfilled course requirements and resulted in the development of wiki content, 
graduate students from Eastern Michigan University this summer studied the publications and resumes of the 
Keynoters, and then interviewed them to ask what they considered their most important contributions to the field. In 
addition, Keynoters were asked to consider the directions the field might be heading and where their future work 
would fit into the developing field. Their contributions to the field have been captured through video interviews, 
writings and multimedia presentations. 

Attendees at the 2010 AECT conference in Anaheim will have the opportunity to create an edu- 
techKNOWiki account and establish their profile and invite their colleagues to participate. Once they have created 
an account, these conference participants can begin creating content for the knowledge base. 

Interactive Learning Communities 

Perhaps the most common way of accessing information in the knowledge base will be through the wiki 
search engine. People may connect with other people in the sociogram by finding their classmates and instructors 
from their graduate programs. The role of the third part of the edu-techKNOWiki is to bring the people and the 
information together to solve problems and provide professional development. 

Conceptual Description 

Conceptually, the third element of the project is to provide a collaborative environment for the 
development of communities of practice surrounding use of educational technology across contexts of practice (PK- 
12 schools, higher education, corporate training, health, military and international settings), and where the context of 
practice is research, a community of research. Wenger (2006) defines communities of practice as: “groups of people 
who share a concern or a passion for something they do and learn how to do it better as they interact regularly.” 
Wenger contrasts community of practice to clubs or networks between people, indicating that a community of 
practice centers around a commitment to a “domain of interest” through which members “build relationships that 
enable them to learn from each other.” Physical proximity is not necessary, but opportunities to problem solve 
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together, share information, experiences and assets are. The first context in which an interface to communities of 
practice in edu-techKNOWiki is developing is P-12 education. 

P-12 schools encourage the infusion of technology into all classrooms, but the level of experience with 
technology and support provided varies widely. Educational technologists fill a variety of roles at the school and 
district levels, but often have limited time with individual teachers. Through the wiki communities of practice, 
teachers and technology facilitators will have access to experts in the field as well as others with common interests 
to engage in problem solving about technology infusion. The learning community page, like other wiki pages, can 
be established by any member by describing the domain of the problems to be discussed. Participants can then use 
the discussion pages for conversation, as well as share links to the knowledge base and other helpful resources. 

Development Process 

As with major development projects, a solution cannot be designed without analysis of the context and 
content. Development of the problem solving interface for the P-12 learning communities began with an analysis 
process based on a modified Delphi technique to identify and categorize the tasks which are performed in schools 
using educational technology. Further analysis applied Activity Theory (Engestrom, 1987, as cited in Marken, 2006) 
a process in which human behavior is used as a basis of task analysis that examines both the tasks and the opposing 
forces present in the human context in which the tasks are performed. Researchers in the development of knowledge 
management systems have found Activity Theory useful as a means for considering culture when developing 
training modules (Schwen, Kalman & Evans, 2006). 

Graduates from the last ten years from the educational technology master’s program in a midwestern 
university who currently work in a P-12 educational environment were invited to participate in a modified Delphi 
process to identify the tasks for which P-12 teachers use educational technology. Participants individually listed all 
of the tasks for which they use technology in the schools. Facilitators compiled the list of more than 50 items into 
loose categories and invited participants to edit and prioritize the categories and re-assign and align the tasks. The 
result was a list of 15 categories of tasks, with those related to use of technology in teaching and learning as the 
highest ranking: 

• Support student learning 

• Teach students about/how to use technology and facilitate their use of technology as a tool 

• Provide professional development 

• Communicate information 

• Promote access for special needs and/or overcome limitation(s) 

• Assess learning 

• Plan instruction 

• Create and deliver visual presentations 

• Develop and deliver distance learning 

• Collect data and perform measurements 

• Manipulate, analyze and interpret data 

• Manage technology resources 

• Organize and store information 

• Plan, draft/design/create, proofread, revise and publish text and graphic representations of information 

• Use hardware in instruction (integrate technology into instruction) 

Once categories were identified, individual interviews were scheduled with participants to discuss their use 
of educational technology in the schools. The interview protocol, developed based on the Eight-Step Model for 
examining a situation (Mwanza, 2001, as cited in Marken, 2006), included the following questions about the eight 
elements of every activity: 

1. Identification of the activity -What sort of activity am I 

interested in? Tools 

2. Identification of the goal (objective) of the activity — Why is 
this activity taking place? What will change as a result of this 
activity? 

3. Identification of the people (subjects) involved in this 
activity — Who is involved in carrying out this activity? 

4. Identification of the tools/processes mediating the activity — 
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By what means are the subjects carrying out this activity? 

5. Identification of the rules and regulations mediating the activity — Are there any cultural norms, rules 
or regulations governing the performance of this activity? 

6. Description of the division of labor mediating the activity — Who is responsible for what, when 
carrying out this activity and how are the roles organized? 

7. Description of the community in which activity is conducted — What is the environment in which this 
activity is carried out? 

8. Identification of the desired outcome - What is the desired Outcome form carrying out this activity? 

In addition to information about the tasks themselves, analysis of responses to these questions can reveal 
contradictions between opposing forces within each element, known as primary contradictions. An example of a 
situation identified in the study, in which an element could contradict itself, was a situation in which a member of a 
technology support team for a vendor, who usually interfaced with the technology facilitators in the school rather 
than directly with teachers, found that budget cuts had forced termination of most technology facilitator roles. Loss 
of these critical roles put the vendor into the role of direct teacher support. The vendor both wanted to lobby for re- 
hire of the technology facilitator positions and wanted to maintain direct support of the teachers. The conflict 
inherent in the situation exists because the vendor’s goals conflict with each other and cannot both be achieved 
within this cultural system. 

To unveil secondary contradictions (between elements rather than within any one element) questions were 
asked that combined the elements. For example: 

• What Tools does the Subjects use to achieve their Objective and how? 

• What Rules affect the way the Subjects achieve their Objective and how? 

• How does the Division of Labor influence the way the Subjects satisfy their Objective? 

• How do the Tools in use affect the way the Community achieves the Objective? 

• What Rules affect the way the Community satisfies their Objective and how? 

• How does the Division of Labor affect the way the Community achieves the Objective? 

Communities of practice develop around problems. The problems occur where contradictions exist within or 
between activities. 



Technical Issues 

Technically the wiki is served from a virtual server and uses MediaWiki, the same software engine that 
drives Wikipedia, arguably the most successful online collaboration project ( http://en.wikipedia.org ) in the world. 

Points of interest will include the variety of ways in which users can enter the system. The most common 
means will be through alphabetical or topical search. The more innovative interface is the problem-based entry, 
based on an Activity Theory analysis of tasks performed by PK-12 educational technologists. Sections accessed 
through this interface serve as the foundation of the PK-12 communities of practice for ongoing teacher professional 
development in technology integration. Future development of similar interfaces is anticipated for educational 
technologists across other practice contexts such as business and industry, higher education, military, health and 
international. 

Future 

The strength of edu-techKNOWiki is that it belongs to the community that comprises the field of 
educational technology; it represents the dynamic, collective thinking of that body of professionals and permits the 
evolution of the record of those thoughts. The ongoing development process must involve faculty and graduate 
students in educational technology programs nationwide as well as practitioners. In addition, in the future we 
foresee a board of directors representing sponsoring organizations and practitioners in all areas of the field. It has 
been suggested that organizations willing to provide server space for the wiki might participate in a rotation to 
support shared ownership of the edu-techKNOWiki. 
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Abstract 

The growth of distance learning in higher education has heightened the need to evaluate the effectiveness online 
degree programs. Twenty-eight educational practitioners pursuing an online Masters of Education degree 
participated on this evaluation study. The analysis of their responses to a 46-item online survey showed the program 
effectiveness; recommendations for improvements on course design and development, assessment strategies and 
learner support were made. 



The Need for this Study 

The increasing growth of online programs in higher education calls for a need to evaluate these programs in a 
systematic way. In this study Fitzpatrick, Sanders & Worthen’s (2011, p. 7) definition of evaluation is used. They 
define evaluation as determining “the value, worth or merit of an evaluation object in relation to a set of criteria.” 
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As cyberlearning becomes a trend, being able to conduct methodologically sound evaluative studies on the impact of 
online programs becomes an important part of the program itself. 

Program evaluation is a vital element of successful distance education programs (Rovai, 2003). It should highlight 
students’ learning contexts and serve as a vehicle for students to voice their recommendations for program 
improvement. Therefore, this study is focused on evaluating the effectiveness of an online Master’s program from 
the students’ or users’ perspective. Online graduate programs should be designed with adult learners in mind to meet 
students' goals, expectations and their learning styles. Do online programs effectively offer courses that 
accommodate student’s needs? To answer this question and others, an evaluation of an online Master’s of Education 
degree program in curriculum and instructional technology was conducted. The results of this evaluation are 
described in this paper. 



Goals of this Study 

The online program evaluated was offered at a major research and land-grant university in the U.S. This degree was 
created to meet the needs of adult learners, especially K-12 teachers, and other educational practitioners, who were 
seeking and an advanced degree in Education, but could not enroll in a residential program. Using a cohort model, 
students enter the program every other year and enroll in one course per semester completing the program in three 
years. In the fifth year (after one cohort had completed the program, one cohort was in its second year, and a third 
cohort was starting the program), a study on the effectiveness of the online M.Ed program from the perspective of 
the distance learning users became a priority. The purpose was to conduct a systematic evaluation to inform 
decisions and actions for improving the quality of the online program. The evaluation addressed the following 
questions: 



Ql: Has the program achieved its purpose? 

Q2: Has the program helped students to define their educational goals? 

Q3: Has the program content met the students’ expectations? 

Q4: Are students applying the knowledge and skills that they have learned in this program into their 
own professional practice? 

Q5: What kind of issues have students encountered while pursuing this program? 

Evaluation Methodology 
The Online Program 

The online Master’s of Education in Curriculum and Instructional Technology (M.Ed in CIT) graduate program 
consists of 9 courses and 32 credits offered in a learning community environment to a cohort of students with a new 
cohort starting every two years. Cohort groups ranged in size from 8 -20 students. Most of the courses were 
delivered using WebCT as the learning management system. A creative component, such as an action research study 
or a portfolio, served as the culminating experience for students to synthesize their learning throughout the program. 

The program uses a blended instructional approach where 85% of the instruction is on-line and 15% is face to face. 
Generally, students enrolled in one course per semester and each course begins with a face-to-face meeting (some 
courses had additional face-to-face meetings during the semester). When feasible, students travel to the university. 
Those who cannot travel, participate via internet conferencing (e.g. Skype, etc.). The face-to-face meeting allows for 
students and instructors to see each other and get to know one another in a non-mediated forum. In these initial 
meetings participants review course expectations, receive essential instruction (best provided face-to-face) and use 
the on-site technology laboratory facilities. Technical support is provided by the program coordinator, who assists 
students in resolving technical problems, assists faculty in the design of online course materials, and serves as a 
communication liaison between faculty and students. At the time of the study, there were six full-time faculty 
members and one administration staff involved in the M.Ed in CIT. 

Students entering the program the same semester are grouped into cohorts. Cohorts have been used effectively in a 
wide variety of educational settings to foster learning (Barnett & Muse, 1993); cohorts are especially important for 
on-line students to quickly acclimate to being physically distant from their peers and instructors. Cohort grouping 
helps to develop and maintain group dynamics across individual classes throughout the program. But to prevent the 
phenomenon of “group think’’ (where everyone is so familiar with each other that new ideas and approaches are 
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rarely introduced, explored, and accepted) 3-5 additional students, not in the cohort, enroll in courses with the cohort 
students. That is to say, students who may be in the traditional face-to-face program or in other degree areas often 
enroll in courses with the online cohort students. 



The Participants 

Twenty-eight students participated in this study from January to December 2009. All students had a background in 
teaching, and most were or had been full time K-12 teachers. Out of the 28 students, four were from the first cohort 
(cohort 1, beginning in 2004), 1 1 from the second cohort (cohort 2, beginning in 2006), and 13 from the third cohort 
(cohort 3, beginning in 2008). 

The four cohort 1 students were females between the ages of 20 and 40. In cohort 2, four students were male and 
seven were female ranging in age from 20 to 50 years old. Cohort 3 had seven males and six females ranging from 
20 to over 50 years old. Additional demographic information about the students who participated in this study is 
summarized in Tables 1, 2 and 3. 

Table 1 : Student Online Survey in Cohort 1 (Demographic Data) 



Category 


Number of Students 
n = 4 


Gender 




Female 


4 


Age 




20-30 


3 




31-40 


1 


Position 




K12 Teacher 


4 



Table 2: Student Online Survey in Cohort 2 (Demographic Data) 




Table 3: Student Online Survey in Cohort 3 (Demographic Data) 



Category 


Number of Students 
n= 13 


Gender 




Male 


7 




Female 


6 


Age 



70 





20-30 


7 




31-40 


3 




41-50 


2 




more than 50 


1 


Position 




K12 Teacher 


6 




Technology Coordinator 


5 




Instructional Designer/Software 
Engineer/T rainer 


2 



Data Collection Method 

An online survey was used to collect the data. It consisted of 46 questions: 4 questions on demographics; 7 open- 
ended questions; and 35 close-ended questions with a 5-point Likert scale ranging from strongly disagree (1) to 
strongly agree (5). The areas addressed on the survey were: (a) program objectives, (b) learning activity experiences, 
(c) delivery of materials, (d) instructor, (e) assessment, and (f) program administration. A pilot study of the survey 
was conducted to support reliability of the scores generated by the online survey items. This survey used some of the 
evaluative areas identified by Rae (2004) and the Michigan State University Virtual University Design and 
Technology (2007). 



Data Analysis 

Quantitative data was analyzed using SPSS. Qualitative data from the open-ended items were analyzed through the 
use of thematic coding and content analysis. 

Results from the Evaluation 

The online masters program evaluated had an 88% persistence and graduation rate for its cohort 1 students 
beginning in 2004, with 7 of the 8 students remaining in the program until the completion of all coursework and 
successfully graduating with a Masters of Education degree. Cohort 2 students, who began in 2006, had a 92% 
persistence and graduation rate with 12 of the 13 students completing all coursework and graduating in Spring 2009 
(Correia et al., 2009). Out of the 17 students initially enrolled in cohort 3 that started in 2008, 14 are expecting to 
graduate in Spring 201 1. The results from the evaluation are organized by cohort and evaluation question. 

Cohort 1 Student Perspectives 

The return rate for students in Cohort 1 was 57% for the online survey. Four (57%) out of seven students who 
graduated responded to the online survey. 

Ql: Has the program achieved its purpose? 

Cohort 1 students responded positively to the survey items related to the program’s purpose with an overall average 
of 4.65 (Table 4). However, the mean score for item 2, 3, and 4 was below average (item 2: m = 4.25, item 3: m = 
4.5, item 4: m =4.5). Students expressed mixed opinions regarding the fact that the program was improving their 
leadership and management skills. Open-ended comments reflected concerns related to balancing full-time work and 
pursuing the online degree program. 



T able 4: Online survey items categorized under Ql (cohort 1). 









Response (%) 








# Items 


Strongly 

Disagree 


Somewhat 

Disagree 


Neutral 


Somewhat 

Agree 


Strongly 

Agree 


n = 4 
(Mean) 


1 This program 
provided me 
with real-life 
experiences 
useful to my 


- 


- 


- 


- 


100% 


5 


profession. 
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Response (%) 








# 


Items 


Strongly 

Disagree 


Somewhat 

Disagree 


Neutral 


Somewhat 

Agree 


Strongly 

Agree 


n = 4 
(Mean) 


2 


I have improved 
my leadership 
skills by 
attending this 


- 


- 


25% 


25% 


50% 


4.25 




program. 














3 


This program 
helped me to 
better 
understand 
learning and 
teaching issues. 


- 


- 


25% 


- 


75% 


4.5 


4 


I improved my 
management 
skills by 
attending this 


- 


- 


25% 


- 


75% 


4.5 




program. 














5 


The program 
was flexible 
enough to 
accommodate 
my needs. 


- 


- 


- 


- 


100% 


5 








Average of Q1 






4.65 



Q2: Has the program helped students to define their educational goals? 

Cohort 1 students were satisfied with their course experiences with an average score of 4.8 (Table 5). They stated the 
following opportunities were available to them: (1) to engage in collaborative activities, (2) to use their own 
professional experiences in course activities, (3) to discover new things, and (4) to synthesize new information; 
however, students were not that positive in relation to the time they had to learn at their own pace, and the 
availability of well-designed social and practical activities (item 3, 8 and 9 were below average). 



Table 5: Online survey items categorized under Q2 (cohort 1). 











Response (%) 








# 


Items 


Strongly 

Disagree 


Somewhat 

Disagree 


Neutral 


Somewhat 

Agree 


Strongly 

Agree 


n = 4 
(Mean) 


1 


There were 
collaborative 










100% 






activities in the 
courses I took. 














2 


I have learned 
from 

collaborating 
with my 
classmates. 


- 


- 


- 


25% 


75% 


4.75 


3 


This program 
offered well- 
designed social 
activities. 


- 


- 


- 


50% 


50% 


4.5 








4 


Activities 
required me to 
use my own 
professional 
experiences. 


- 


- 


- 


- 


100% 


5 
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Response (%) 








# 


Items 


Strongly 


Somewhat .. > 


Somewhat 


Strongly 


n = 4 


Disagree 


Disagree 


Agree 


Agree 


(Mean) 


5 


There were 
practical 
activities in this 
program. 


- 


- 


- 


100% 


5 


6 


I had many 
opportunities to 
discover new 
things in this 
program. 


- 


- 


- 


100% 


5 


7 


Course 
activities 
encouraged me 
to synthesize 


_ 


_ 


_ 


100% 


5 




new 














information. 












8 


In this program, 
practical 
activities were 
well-prepared. 


- 


- 


50% 


50% 


4.5 


9 


I was given 
enough time to 
learn at my own 
pace during the 


- 




50% 


50% 


4.5 




program. 


















Average of Q2 






4.8 



Q3: Has the program content met the students ’ expectations? 

Most of cohort 1 students agreed that the program content met their expectation with an average score of 4.5(Table 
6). However, open and close-ended questions showed some concerns about adequate evaluation activities to assess 
their own learning and opportunities to assess themselves in every learning activity. Mean scores for items 10 and 1 1 
in Table 6 were significantly below the average (item 10: m = 3.5, item 11: m = 3). Students recommended 
additional and timely feedback from the course instructors about students’ progress. 



Table 6: Online survey items categorized under Q3 (cohort 1). 











Response (%) 








# 


Items 


Strongly 


Somewhat 


Neutral 


Somewhat 


Strongly 


n= 11 


Disagree 


Disagree 


Agree 


Agree 


(Mean) 


1 


I was able to obtain 
course materials 
easily via a variety 
of technological 
tools. 


- 


- 


- 


- 


100% 


5 


2 


I was able to work 










100% 






in WebCT. 










J 


3 


Course materials 
were well- 
designed. 


- 


- 


- 


- 


100% 


5 


4 


Course materials 
were motivating. 


- 


- 


25% 


50% 


25% 


4 


5 


Delivery of course 
materials were 
suitable for me. 


- 


- 


- 


- 


100% 


5 
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Response (%) 








# 


Items 


Strongly 

Disagree 


Somewhat 

Disagree 


Neutral 


Somewhat 

Agree 


Strongly 

Agree 


n= 11 
(Mean) 


6 


Instructors had 
good knowledge of 
subject. 


- 


- 


- 


- 


100% 


5 


7 


Instructors 
organized course 
well. 


- 


- 


- 


- 


100% 


5 


8 


Instructors were 










100% 


£ 




well prepared. 












9 


I was able to easily 
communicate with 
instructors. 


- 


- 


- 


- 


100% 


5 



10 There were 
adequate 



evaluation - 25% 25% 25% 25% 3.5 

activities to asses 

my own learning. 



11 


I was able to assess 
myself in each 
activity I 
participated. 


50% 


- 


50% 


- 


3 


12 


Student assessment 
strategies enhanced 
my learning. 


25% 


- 


25% 


50% 


4 


13 


Grading was 
accurate. 


25% 


- 


25% 


50% 


4 






Average of Q3 








4.5 



Q4: Are students applying the knowledge and skills that they have learned in this program into their own 
professional practice? 

Cohort 1 students were clearly able to utilize knowledge/skills gained from the program in their own professional 
practice (Table 7). Open-ended questions revealed students were satisfied with this aspect of the online graduate 
program. 



Table 7: Online survey items categorized under Q4 (cohort 1). 







Response (%) 








# Items 


Strongly 

Disagree 


Somewhat .. x . 

Neutral 

Disagree 


Somewhat 

Agree 


Strongly 

Agree 


n= 11 
(Mean) 


1 I could apply 
what have 
learned in this 
program to my 
profession. 


- 


- 


- 


100% 


5 






Average of Q4 






5 



Q5: What kind of issues have students encountered while pursuing this program? 

Students were asked to rate the administration of the program, which related to the program length and course 
sequence, support received, and educational cost and value. This aspect received an average rating of 4. 75 (Table 8). 
It seems that the program administration was not an issue for any of the students and that they received adequate 
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administrative and technology support throughout the program. However, items 6 and 7 were rated below the 
average, which shows that the students were likely not fully pleased cost and value of the overall program. 



Table 8: Online survey items categorized under Q5 (cohort 1). 











Response (%) 








# 


Items 


Strongly 

Disagree 


Somewhat 

Disagree 


Neutral 


Somewhat 

Agree 


Strongly 

Agree 


n= 11 
(Mean) 


1 


The length of this 
program is adequate to 
complete a Master’s 
Degree. 


- 


- 


- 


- 


100% 


5 


2 


The courses I took were 










100% 






logically sequenced. 










J 



3 I was given appropriate 
administrative support 



when I had questions - 100% 5 



(admission, add/drop 
courses, tuitions/fees) 



4 


My major professor 
encouraged me to pursue 
my area of interest. 


- 


- 


- 


100% 


5 


5 


I was given appropriate 
support when I had any 
course -related 
technology problems. 


- 


- 


- 


100% 


5 


6 


The program cost as a 
whole was reasonable. 


- 


25% 


50% 


25% 


4 


7 


The value of education 
outweighed the program 
cost. 


- 


- 


75% 


25% 


4.25 






Average of Q5 








4.75 



Cohort 2 Student Perspectives 

The return rate for students in Cohort 2 was 92% for the online survey. Eleven (92%) out of twelve students who 
graduated responded to the online survey used for this study. 

Ql: Has the program achieved its purpose? 

Overall, cohort 2 students agreed that the objectives of the program were met. It is shown in Table 9 that evaluative 
question 1 received overall average of 4.18. According to students, the program helped them to improve leadership 
skills. It provided real-life experiences for their profession. However, the mean score for item 3, 4, and 5 in Table 9 
was below average (item 3: m = 4.09, item 4: m = 3.82, item 5: m =4.09). The findings indicate that students found 
the management skills and enhancing understanding of their job after engaged in this program were appropriate, but 
the course content is still need to be improved. Therefore, it is likely that some of the coursework might not really 
applicable to their teaching and enhancing their management skills. They commented that they wanted to learn more 
about the use of technology in the courses, which may contribute positive effect on their own teaching and 
classroom. 



Table 9: Online survey items categorized under Q2 (cohort 2). 









Response (%) 








# Items 


Strongly 

Disagree 


Somewhat 

Disagree 


Neutral 


Somewhat 

Agree 


Strongly 

Agree 


n= 11 
(Mean) 


1 This program 
provided me 








63.63% 


36.37% 


4.36 


with real-life 
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Response (%) 








# 


Items 


Strongly 

Disagree 


Somewhat 

Disagree 


Neutral 


Somewhat 

Agree 


Strongly 

Agree 


n= 11 
(Mean) 


experiences 
useful to my 
profession. 


2 


I have improved 
my leadership 
skills by 
attending this 
program. 


- 


- 


- 


45.45% 


54.55% 


4.55 


3 


This program 
helped me to 
better 
understand 
learning and 
teaching issues. 


- 


9.0% 




63.63% 


27.28% 


4.09 


4 


I improved my 
management 
skills by 
attending this 
program. 


- 


9.09% 


9.09% 


72.73% 


9.09% 


3.82 


5 


The program 
was flexible 
enough to 
accommodate 
my needs. 


- 


18.18% 


18.18% 




63.64% 


4.09 








Average of Q1 






4.18 



Q2: Has the program helped students to define their educational goals? 

The students were asked to assess the learning activities that they had experienced in order to set their educational 
goals. Cohort 2 students were satisfied with their course experiences and the average score of satisfaction was 4.42 
(Table 10). They indicated that the program structure was flexible and allowed them to work from home, adjust 
teaching schedules as well as family commitments; however, one student responded that they were not given enough 
time to learn at their own pace during the program (reflects from item 9 was below average, item 9: m = 4.36). 
Students really appreciated being a cohort group as they learned as much from other students in the group; they were 
together throughout the program, developed a strong relationship, and could rely on each other for assistance and 
guidance. The program also provided them with opportunities to get involved in collaborative activities, discover 
new things, and synthesize new information, which they could connect those experiences into their own classroom 
teaching. It can be said that the program helps students to set their educational goals through providing the flexibility 
of the program structure, schedule, and cohort system. 



Table 10: Online survey items categorized under Q2 (cohort 2). 











Response (%) 








# 


Items 


Strongly 

Disagree 


Somewhat 

Disagree 


Neutral 


Somewhat 

Agree 


Strongly 

Agree 


n= 11 
(Mean) 


1 


There were 
collaborative 
activities in the 
courses I took. 


- 


- 


- 


36.36% 


63.64% 


4.64 


2 


I have learned 
from 

collaborating 
with my 
classmates. 


- 


- 


18.18% 


9.09% 


72.73% 


4.55 
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Response (%) 








# 


Items 


Strongly 


Somewhat 


Neutral 


Somewhat 


Strongly 


n= 11 


Disagree 


Disagree 


Agree 


Agree 


(Mean) 


3 


This program 
offered well- 
designed social 
activities. 


- 


- 


18.18% 


45.45% 


36.37% 


4.18 


4 


Activities 
required me to 
use my own 
professional 
experiences. 


- 


- 


- 


63.63% 


36.37% 


4.36 


5 


There were 
practical 
activities in this 
program. 


- 


- 


- 


63.63% 


36.37% 


4.36 


6 


I had many 
opportunities to 
discover new 
things in this 
program. 


- 


- 


9.09% 


54.54% 


36.37% 


4.27 


7 


Course 
activities 
encouraged me 
to synthesize 


_ 


_ 


_ 


27.27% 


72.73% 


4.73 




new 
















information. 














8 


In this program, 
practical 
activities were 
well-prepared. 


- 


- 


- 


63.63% 


36.37% 


4.36 


9 


I was given 
enough time to 
learn at my own 
pace during the 


9.09% 






27.27% 


63.64% 


4.36 




program. 




















Average of Q2 








4.42 



Q3: Has the program content met the students ’ expectations? 

Students were asked to evaluate several aspects of the program related to program content, which including course 
materials, learning support, and assessment. Students were satisfied with these aspects and the average score of 
satisfaction was 4.32 (Table 11). This indicates that the element of TPACK (Technology, Pedagogical and Content 
Knowledge) was embedded in the course contents. Although the score was high, students commented that they 
should have been exposed to more technology tools that could be used in their classroom. It is shown that it is good 
to have an alternative to WebCT and that it may be better to use contemporary technology tools, so that students are 
able to work with sophisticated tools in order to enhance their technological skills. 

On the other hand, the mean scores for items 10, 11, and 12 in Table 1 1 were below than category ‘Agree ’ (item 10: 
m = 3.91, item 11: m = 3. 73 item 12: m = 3.91). Most students indicated that the course materials were easy to 
access and well prepared, but be motivating and consistent in courses organization. 

Adequate and timely feedback from instructors is essential in any teaching and learning setting. Students reported 
that most of the instructors were very experienced and knowledgeable, and cooperative. However, some students 
felt that they did not receive adequate and timely feedback from instructors. They commented that high quality 
rubrics should be designed for each assessment in order for students to examine their learning outcome. Thus, it is 
noted that instructors need to be more clear and precise about their expectation and to also provide a platform for 
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students to ask questions and interact with others. 



Table 11: Online survey items categorized under Q3 (cohort 2), 











Response (%) 








# 


Items 


Strongly 

Disagree 


Somewhat 

Disagree 


Neutral 


Somewhat 

Agree 


Strongly 

Agree 


n= 11 
(Mean) 


1 


I was able to obtain 
course materials 
easily via a variety 
of technological 
tools. 


- 


- 


- 


27.27% 


72.73% 


4.73 


2 


I was able to work 
in WebCT. 


- 


- 


- 


- 


100% 


5.00 


3 


Course materials 
were well- 
designed. 


- 


- 


- 


72.72% 


27.28% 


4.27 


4 


Course materials 
were motivating. 


- 


- 


- 


90.91% 


9.09% 


4.09 


5 


Delivery of course 
materials were 
suitable for me. 


- 


- 


9.09% 


63.63% 


27.28% 


4.18 


6 


Instructors had 
good knowledge of 
subject. 


- 


- 


- 


9.09% 


90.91% 


4.91 


7 


Instructors 
organized course 
well. 


- 


- 


- 


81.81% 


18.19% 


4.18 


8 


Instructors were 
well prepared. 


- 


- 


- 


72.72% 


27.28% 


4.27 


9 


I was able to easily 
communicate with 
instructors. 


- 


- 


- 


45.45% 


54.55% 


4.55 


10 


There were 
adequate 
evaluation 
activities to asses 
my own learning. 


- 


18.19% 


9.09% 


36.36% 


36.36% 


3.91 


11 


I was able to asses 
myself in each 
activity I 
participated. 


- 


18.19% 


9.09% 


54.54% 


18.18% 


3.73 


12 


Student assessment 
strategies enhanced 
my learning. 


- 


18.19% 


9.09% 


36.36% 


36.36% 


3.91 


13 


Grading was 
accurate. 


- 


- 


- 


54.54% 


45.46% 


4.45 








Average of Q3 








4.32 



Q4: Are students applying the knowledge and skills that they have learned in this program into their own 
professional practice? 

Cohort 2 students felt they applied their learning to their own profession and were able to add technology into their 
own curriculum effectively. They evaluated these aspects relatively high with a mean of 4.64 (Table 12). Most of 
the comments focused on gaining research skills and the ability to integrate new technology into their profession. 
The students were mostly pleased with the skills that they have learned from this program such as project based 
learning with technology integration and strategy of metacognition. 
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Table 12: Online survey items categorized under Q4 (cohort 2) 







Response (%) 








# Items 


Strongly 

Disagree 


Somewhat .. x . 

Neutral 

Disagree 


Somewhat 

Agree 


Strongly 

Agree 


n= 11 
(Mean) 


1 I could apply 
what I have 
learned in this 


3.5% 




36.36% 


63.64% 


4.64 


program to my 
profession. 
















Average of Q4 






4.64 



Q5: What kind of issues have students encountered while pursuing this program? 

Students were asked to rate the administration of the program, which related to the program length and course 
sequence, support received, and educational value. This aspect received average ratings of 4.34 (Table 13). It seems 
that the program administration was not an issue for most students and that they received adequate administrative 
and technology support throughout the program. However, items 6 and 7 were rated below the average ( item 2: m = 
4.27, item 6: m = 3.57, item 7: m = 4.18). It shows that the students were likely not satisfied with the sequence 
course taken in the program and also the program cost. 



Table 13: Online survey items categorized under Q5 (cohort 2). 











Response (%) 








# 


Items 


Strongly 

Disagree 


Somewhat 

Disagree 


Neutral 


Somewhat 

Agree 


Strongly 

Agree 


n= 11 
(Mean) 


1 


The length of this 
program is adequate to 
complete a Master’s 
Degree. 


- 


- 


- 


18.18% 


81.82% 


4.82 


2 


The courses I took were 
logically sequenced. 


- 


- 


- 


72.72% 


27.28% 


4.27 



3 I was given appropriate 
administrative support 



when I had questions - - 9.09% 18.18% 72.73% 4.64 



(admission, add/drop 
courses, tuitions/fees) 



4 


My major professor 
encouraged me to pursue 
my area of interest. 


- 


27.27% 


9.09% 


63.64% 


4.36 


5 


I was given appropriate 
support when I had any 
course-related technology 
problems. 


- 


9.09% 


27.27% 


63.63% 


4.55 


6 


The program cost as a 
whole was reasonable. 


27.27% 


9.09% 


45.45% 


18.19% 


3.55 


7 


The value of education 
outweighed the program 
cost. 


9.09% 


9.09% 


36.36% 


45.46% 


4.18 






Average of Q5 








4.34 
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Cohort 3 Student Perspectives 

The return rate for students in Cohort 3 was 93% for the online survey. Thirteen (93%) out of fourteen students who 
are planning to graduate in spring 2011 responded to the online survey used for this evaluative study. 

Ql: Has the program achieved its purpose? 

Overall, cohort 3 students agreed that the objectives of the program were met with an overall average of 4.62 (Table 
14). According to students, the program provided real-life experiences, helped them to better understand learning 
and teaching issues and was flexible enough to accommodate their needs. However, the mean score for item 2 and 4 
in Table 14 was below average, which displays some areas of program improvement on developing leadership and 
management skills. Open-ended questions also included positive comments (flexibility, application and pace) to the 
program and its objectives, but some concerns/ issues were raised: (1) inconsistency on participation and strategies 
for engagement on online discussions, (2) need for more lecture-style presentations (pre-recorded live), and (3) 
inconsistency on work load among the different courses and design of the online environment. 



T able 14: Online survey items categorized under Ql (cohort 3). 











Response (%) 








# 


Items 


Strongly 

Disagree 


Somewhat 

Disagree 


Neutral 


Somewhat 

Agree 


Strongly 

Agree 


n= 13 
(Mean) 


1 


This program 
provided me 
with real-life 
experiences 
useful to my 
profession. 


- 


- 


7.7% 


38.5% 


53.8% 


4.69 


2 


I have improved 
my leadership 
skills by 
attending this 


- 


7.7% 


- 


61.5% 


30.8% 


4.38 




program. 














3 


This program 
helped me to 
better 
understand 
learning and 
teaching issues. 


- 


- 


- 


46.2% 


53.8% 


4.77 


4 


I improved my 
management 
skills by 
attending this 


- 


- 


15.4% 


46.2% 


38.5% 


4.46 




program. 














5 


This program 
was flexible 
enough to 
accommodate 
my needs. 


- 


- 


7.7% 


46.2% 


46.2% 


4.77 








Average of Ql 






4.62 



Q2: Has the program helped students to define their educational goals? 

Overall cohort 3 students were satisfied with their course experiences with an average score of 4.35 (Table 15). 

They reported opportunities to work collaboratively in class and activities where they were able to apply their 
professional experiences, encouraged to discover new things and synthesized new information. Lower scores (below 
average) were shown regarding the offering of social activities, design of practical activities and enough time to 
learn at their own pace (items 3, 8 and 9 with m = 4.08, m=4.23 and m=3.69, respectively). 
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Table 15: Online survey items categorized under Q2 (cohort 3). 









Response (%) 








# 


Items 


Strongly 

Disagree 


Somewhat XT . 

Neutral 

Disagree 


Somewhat 

Agree 


Strongly 

Agree 


n= 13 
(Mean) 


1 


There were 
collaborative 
activities in the 
courses I took. 


- 


- 


46.2% 


53.8% 


4.54 


2 


I have learned 
from 

collaborating 
with my 
classmates. 


- 


- 


23% 


76.9% 


4.77 


3 


This program 
offered well- 
designed social 
activities. 


- 


23% 


46.2% 


30.8% 


4.08 


4 


Activities 
required me to 
use my own 
professional 
experiences. 


- 


- 


46.2% 


53.8% 


4.54 


5 


There were 
practical 
activities in this 
program. 


- 


- 


53.8% 


46.2% 


4.46 


6 


I had many 
opportunities to 
discover new 
things in this 
program. 


- 


15.4% 


30.4% 


53.8% 


4.46 


7 


Course 
activities 
encouraged me 
to synthesize 


_ 


_ 


53.8% 


46.2% 


4.46 




new 














information. 












8 


In this program, 
practical 
activities were 
well-prepared. 


- 


7.7% 


53.8% 


30.8% 


4.23 


9 


I was given 
enough time to 
learn at my own 
pace during the 


- 


15.4% 23% 


38.5% 


23% 


3.69 




program. 


















Average of Q2 






4.35 



Q3: Has the program content met the students ' expectations? 

Students were asked about their satisfaction in relation to course materials, learning support, and assessment with an 
average score of 4.20 (Table 16). Even though access to course materials in WebCT, appropriateness of the mode of 
delivery and instructors’ technology, pedagogical and content knowledge were scored above the average, there were 
several aspects of the program content that had lower scores (rn< 4.90). They were: 

• the overall design of the course materials {item 3: m=3.86), 
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• the motivational design of the course materials {item 4: m=3.85 ) 

• communication with the instructors {item 9: m=4.15 ), and 

• evaluation activities, assessment and grading (item 10: m = 3.92, item 11: m = 4.00, item 12: m = 4.00, 
item 13: m=4.08). 



Table 16: Online survey items categorized under Q3 (cohort 3). 











Response (%) 








# 


Items 


Strongly 

Disagree 


Somewhat 

Disagree 


Neutral 


Somewhat 

Agree 


Strongly 

Agree 


n= 13 
(Mean) 


1 


I was able to obtain 
course materials 
easily via a variety 
of technological 
tools. 


- 


- 


- 


76.9% 


23.1% 


4.23 


2 


I was able to work 
in WebCT. 


- 


- 


- 


46.2% 


53.8% 


4.54 


3 


Course materials 
were well- 
designed. 


- 


- 


7.7% 


69.2% 


23.1% 


4.15 


4 


Course materials 
were motivating. 


- 


- 


23.1% 


69.2% 


7.7% 


3.85 


5 


Delivery of course 
materials were 
suitable for me. 


- 


- 


7.7% 


61.5% 


30.8% 


4.23 


6 


Instructors had 
good knowledge of 
subject. 


- 


- 


15.4% 


46.2% 


38.5% 


4.85 


7 


Instructors 
organized course 

well. 


- 


- 


7.7% 


53.8% 


38.5% 


4.31 


8 


Instructors were 
well prepared. 


- 


- 


- 


76.9% 


23.1% 


4.23 


9 


I was able to easily 
communicate with 
my instructors. 


- 


- 


7.7% 


69.2% 


23.1% 


4.15 


10 


There were 
adequate 
evaluation 
activities to asses 
my own learning. 


- 


7.7% 


23.1% 


38.5% 


30.8% 


3.92 


11 


I was able to asses 
myself in each 
activity I 
participated. 


- 


- 


23.1% 


53.8% 


23.1% 


4.00 


12 


Student assessment 
strategies enhanced 
my learning. 


- 


- 


23.1% 


53.8% 


23.1% 


4.00 


13 


Grading was 
accurate. 


- 


- 


7.7% 


76.9% 


15.4% 


4.08 








Average of Q3 








4.20 



Q4: Are students applying the knowledge and skills that they have learned in this program into their own 
professional practice? 

Students reported that they applied the knowledge and skills they were learning in the program into their 
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professional practices with an average score of 4.46 (Table 17). Most of the comments focused on the opportunities 
to use different applications in the classroom and to effectively use technology for instruction. Other students 
mentioned the application of instructional design principles into their practices; with other students explaining that 
they were using what they were learning in the program into professional development activities. 



Table 17: Online survey items categorized under Q4 (cohort 3) 







Response (%) 








# Items 


Strongly 

Disagree 


Somewhat XT . , 

Neutral 

Disagree 


Somewhat 

Agree 


Strongly 

Agree 


n= 13 
(Mean) 


1 I could apply 
what I have 
learned in this 
program to my 
profession. 




- 


38.4% 


53.8% 


4.46 






Average of Q4 






4.46 



Q5: What kind of issues have students encountered while pursuing this program? 

Cohort 3 students rated a variety of features of the program regarding the administration, program length and course 
sequence, support received, and educational value of the program. This aspect received average ratings of 4.00 
(Table 18), with items 4 and 6 rated below that. This shows that the students were not fully satisfied with the 
encouragement they got from their major professor on pursuing an area of interest ( m=3.31 ) and with the cost of the 
program as a whole ( m=3.69 ). Two of the major issues pointed out by the students were on the open-ended 
questions: (1) keeping self-motivation, (2) willingness to self-discipline and self-monitor; and (3) inconsistency 
among instructors in course planning, instructional materials development, course communications, and assessment. 



Table 18: Online survey items categorized under Q5 (cohort 3). 











Response (%) 








# 


Items 


Strongly 

Disagree 


Somewhat 

Disagree 


Neutral 


Somewhat 

Agree 


Strongly 

Agree 


n= 13 
(Mean) 


1 


The length of this 
program is adequate to 
complete a Master’s 
Degree. 


- 


- 


7.7% 


53.8% 


30.8% 


4.23 


2 


The courses I took were 
logically sequenced. 


- 


- 


7.7% 


69.2% 


23.1% 


4.15 


3 


I was given appropriate 
administrative support 
when I had questions 
(admission, add/drop 
courses, tuitions/fees) 


- 


- 


23.1% 


38.5% 


38.5% 


4.15 


4 


My major professor 
encouraged me to pursue 
my area of interest. 


7.7% 


- 


53.8% 


30.8% 


7.7% 


3.31 


5 


I was given appropriate 
support when I had any 
course-related technology 
problems. 


- 


- 


7.7% 


61.5% 


30.8% 


4.23 


6 


The program cost as a 
whole was reasonable. 


- 


15.4% 


23.1% 


38.5% 


23.1% 


3.69 


7 


The value of education 
outweighed the program 


- 


- 


23.1% 


53.8% 


23.1% 


4.00 
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Response (%) 








# Items 


Strongly Somewhat 

Disagree Disagree 


Neutral 


Somewhat 

Agree 


Strongly 

Agree 


n= 13 
(Mean) 


cost. 




Average of Q5 








4.00 



Conclusions and recommendations 

Overall, the participants of the online Masters of Education in Curriculum and Instruction were positive about the 
program. From the users’ perspective the program had achieved its purpose of providing a high quality online 
graduate program in education. A large majority of students completed the program or were on schedule to 
complete the program in the 3-year timeframe. Although the students were positive about the program in general, 
they offered relevant suggestions for program improvement as well as pointing out components of the program that 
supported its effectiveness. Student comments included the following: 

• “I don't know if the workload for this program is any different than the workload for other Masters 
programs, but the discussions we have online definitely get me participating more than I would in a normal 
classroom setting. I am also able to bring my classroom experiences into the learning environment and get 
useful feedback from colleagues.” 

• Keep the cohort system as it provides a sense of belonging among the students and develops strong 
relationships 

• Expand into an online MS and/or Ed.D/Ph.D. 

• Courses should offer more opportunities to engage with the latest technologies for learning and teaching 

• Assessment methods and criteria should be made more explicit through examples of deliverables, 
homework, and discussion replies 

• Coursework should be more aligned with the portfolio standards plus provide additional support in creating 
portfolio. 

• “Push the idea of 'anytime, anywhere' through the fusion of on-line & on-campus instruction. Also, the 
collaborative & collegial atmosphere found in the Cohort would be attractive to most applicants. If 
someone is interested in this field, they would like to see the variety of instructor interests. . .” 

The demands of a technology-saturated society require that degree programs at all levels be offered via distance 
education. The results of this study indicate that offering an online graduate degree in education with an emphasis 
in instructional technology can be an effective means for K-12 teachers to expand their educational background and 
technology integration skills and knowledge. In this study, students who were in or had recently completed an 
online graduate degree were positive overall about their online learning experiences in the program. Greater insight 
into their views at various stages throughout the three-year graduate program will allow instructors and instructional 
designers to better develop environments that meet their situational needs and learning demands. Further research in 
this area is appropriate and needed. 
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Abstract 

Current research (Chang & Smith, 2008; Herbert, 2006; Kemp, 2002) indicates there is a close connection between 
online student satisfaction and retention in this delivery modality. Student dissatisfaction with online courses is often 
the result of a lack of interaction (Bollinger & Martindale, 2004). In this study, 255 student end-of-course 
evaluations for 23 online courses and 17 instructors were evaluated to determine teacher effectiveness in utilizing 
course tools that support interactivity. Results indicated that students want instructors who are active participants, 
provide clear directions and expectations, and are available. This information was then used to create a plan for 
increasing instructor-student and student-student interaction in online courses. The intervention strategies were 
implemented and tested and the student end-of-course evaluations indicated that students found the increased 
communication engaging and motivating resulting in a more learner-centered environment. 



While distance education continues to be a valuable asset in the delivery of course content, research shows 
that interaction (Wanstreet, 2006) impacts student learning and satisfaction. Perez (2001) states that students often 
report dissatisfaction due to the lack of personal interaction between the instructor and students and among students. 
As the competition for online students grows, so do efforts to attract and retain students, particularly since the 
retention rates in online classes have historically been low, (Herbert, 2006). 

Current research in online education suggests a close tie between online student satisfaction and retention 
(Chang & Smith, 2008; Herbert, 2006; Kemp, 2002). Bollinger and Martindale (2004) suggest that student 
satisfaction with online courses is influenced by three major constructs; instructor variables, technical issues, and 
interactivity. They also note that the instructor is the main predictor of student satisfaction with online courses. 

Background and Problem 

One obvious way to remedy the problem of low or infrequent interaction is by increasing personal 
interaction during the course. While this is an obvious solution, the implementation is the challenge. Course 
Management Systems (CMS) provide a variety of tools, that when utilized, have the potential to increase 
communication and learning. The research of Zemsky and Massey (2004) supports the impact of how the instructor 
delivers the course content. They found that although instructors were using online learning delivery methods, they 
were still using teacher-centered pedagogy. Mahle clearly states that, “Instructors need to be cognizant of 
incorporating a significant amount of interactivity into their courses (Mahle, 2007, p. 47). The question then 
becomes how to get course instructors to change their pedagogy and integrate more interaction in their online 
courses. 

Given the need to help fulltime and adjunct faculty transition successfully into the expanding online 
environment, this study provides an analysis of 255 end-of-course student evaluations collected from 23 online 
course sections and 17 instructors over a period of six months in 2010. Current research on best practices in online 
education formed the theoretical base for the study which involved graduate students enrolled in one of the required 
core classes offered through the Teacher Education Department in the School of Education. Using the current 
institutionally-approved online end-of-course evaluation of teaching instrument that is contained within the 
University’s existing e-College platform, the purpose of this study was to explore the factors contributing to low 
student evaluations of online faculty and utilize the results to develop effective intervention strategies designed to 
improve the pedagogy and thus, the student ratings of low-scoring faculty. 
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The Study and the Purpose 



One way to impact pedagogy is to first make the instructor aware of the issue. In order to determine the 
level of student satisfaction, 255 online end-of- course student evaluations were analyzed for 23 sections and 17 
instructors over a six-month period from January 2010 through June 2010 at a private, non-traditional, not-for-profit 
university. This study looked at the satisfaction ratings from students completing the course and identified areas of 
need. The assessment of teaching section of the course evaluation revealed three major areas for improvement: 
meaningful chat sessions, active instructor participation in the course, and timely and helpful feedback. The author 
then reviewed student comments to gain a greater insight into student needs. The detailed comments supported 
students’ desire for greater instructor interaction and involvement in the courses including the use of the 
Announcements, Discussion Boards, Virtual Office, and Live Chat. Based on this data, the author searched for 
effective strategies to increase student activity. The author met with colleagues who successfully used instructor- 
student interaction techniques resulting in higher student satisfaction as indicated by their end-of-course evaluations. 
Then, a plan was developed for implementing new strategies to increase interactivity in online courses. 

Theoretical Framework 

The importance of interactive, student-centered instruction has been a central theme in higher education 
since the original Chickering and Gamson (1987) study on The 7 Principles for Good Practice in Undergraduate 
Education. Subsequently updated for distance education in 1996 by Chickering and Ehrmann, Implementing the 7 
Principles: Technology’ as Lever has strongly influenced the development of contemporary research related to best 
practices and effective virtual classroom instructional strategies for use in the online environment. 

Research in best practices for online education emphasizes the importance of promoting interactivity, 
encouraging student-instructor and student-student interchanges as well as building online learning communities 
(Bangert, 2005; Bannan-Ritland, 2002; Dennen, Darabi, & Smith, 2007; Kennedy, 2004). Recent contributions to 
the field of web-based distance education state that interactivity and communication are key components required 
for successful online teaching and learning (Fabry & Schubert, 2009; Mahle, 2007; Moore, 2001; Tobin, 2004). 
Citing results from a recent study on the importance of interaction to student learning within web-based online 
learning programs, Sher (2009) notes that, “Student-instructor interaction and student-student interaction were found 
to be significant contributors of student learning and satisfaction” (p. 102). 

The literature related to interactivity in online learning generally focuses on the traditional trilogy of 
interaction which includes (a) Learner-Content, (b) Learner-Learner, and (c) Learner-Instructor (Chang & Smith 
2009; Moore, 2001). Mahesh and Mclsaac (1999) took a slightly different focus on the same theme of interaction, 
but focused more on communication. Their research study looked at the dynamic of instructor-student 
communication and the strategies the instructor implemented that encouraged communication within the virtual 
classroom, including regular feedback. Communication in online classes most frequently takes the form of 
asynchronous threaded discussions, announcements, virtual office, synchronous chats and e-mail. Instructor 
feedback on course assignments also plays a crucial role in student success and can be regarded as a specific form of 
communication. Effective interaction and successful communication include the use of multiple strategies and 
activities where instructors provide feedback that is both immediate and frequent (Bollinger & Martindale, 2004; 
Dennen, Darabi, & Smith, 2007). 

Research related to interactivity in online instruction formed the basis for this study that focused on 
determining how the interactivity variables presented faculty with the most challenges in relation to student 
satisfaction with instruction. By identifying the key variables, the author sought to develop effective strategies 
designed to improve faculty interaction in specific areas of need. 

Creating and Implementing a Plan to Increase Interaction 

Data from the end-of-course evaluations clearly revealed that students want an instructor who is an active 
participant in the class, provides meaningful synchronous and asynchronous chat sessions, and responds in a timely 
manner. After analyzing the data, the author created a list of interaction strategies based on the research in effective 
online teaching. One section of the online course was taught in July 2010 integrating the interaction strategies. 

[Note: The non-traditional format of the university is a one-month long course that is fast-paced and rigorous.] 
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Given the data showing student areas of concern, the plan for increasing interaction focused on five tools: E-mail, 
Announcements, the Virtual Office, Threaded Discussions, and Synchronous Chat. 

E-mail can often become an afterthought in online classes; however, it has the capability to enhance 
interaction. Sending a welcome e-mail to the class before the class begins sets the tone for communication. In the 
first e-mail contact students were provided with a welcoming statement, a brief introduction to the professor, the 
preferred method of contacting the instructor, the course syllabus/outline, and a request for a response to the e-mail. 
Several strong messages are conveyed through this first communication: you care, you are present, and you are 
available. Additional e-mails were utilized throughout the course to remind students of course events, clarify 
questions, and share resources. 

The Announcements tool has the potential to provide multiple interactions with students. If used only 
infrequently and in a pedantic manner; this tool is not being used effectively. To increase instructor-student 
interaction, Announcements should be of high interest and posted at regular and frequent intervals. Announcements 
were used to provide general information to the class, as well as tips and suggestions for successfully completing 
assignments. The content of six to eight announcements each week ranged from reminding students of when 
assignments were due, where to locate specific information in the course, how to locate a peer-reviewed article in 
the online library, and where to find student examples of assignments. Announcements were often the result of 
questions e-mailed to the instructor. 

The Virtual Office offers opportunities for student-to-instmctor and student-to-student communication. The 
Announcements tool was used to direct students to the Virtual Office. As a cross-reference the Virtual Office hours 
were posted in Announcements and participation was encouraged. Three times each week the instructor logged on 
and held Virtual Office hours, during which time students could log on, ask questions, and provide feedback. 
Students were able to respond to each other, as well as the instructor. Throughout the week, the Virtual Office was 
checked both in the morning and evening for other questions, comments, and feedback. 

Discussion Boards, asynchronous chat areas, can become difficult to manage when 35 students are required 
to post an initial response and then respond to at least two of their classmates. The instructor used the tool to create 
smaller subgroups that were more user-friendly. Students were invited to share their responses by grade levels in the 
course. Elementary, middle, and high school teachers had their own discussion areas. The instructor responded to 
each initial posting for every student and encouraged participation using carefully crafted question to elicit higher 
order thinking and to help learners make deeper connections. 

The final tool used to increase interactivity was the synchronous chat, Class Live Pro. This tool has both 
audio and video capabilities as well as a plethora of functions. Live chats were conducted once each week. The 
topics were application based. Students were asked to take the discussion thread topic and share how they could 
implement it in their classroom. During the chat, breakout rooms allowed students to work in small groups and then 
reconvene to report out in the large group. 



Results and Recommendations 

The End-of-Course Evaluation for the July class indicated a positive response from students concerning 
interaction. The increased use of Announcements resulted in fewer one-on-one questions from individual students. 
While this may sound like it was counter-productive, it was not. It allowed for more time and interaction in the 
Virtual Office. Students said that they liked knowing there were specific times they could speak with the instructor 
and their classmates. 

The interaction during the Discussion Boards was frequent and enthusiastic. Students reported that they 
liked knowing the instructor was an active participant. Students said that they liked being challenged to make new 
connections. This same response was given concerning the live chats. Students felt the chats provided them with 
authentic solutions to their classroom problems. They also felt more connected to classmates and the instructor. 

This study and the results are limited to one class delivered online with 35 students. Additional studies will 
need to be conducted with faculty who are willing to learn and implement new interactive knowledge and skills. 




One caveat need to be noted: Providing opportunities for increased student interaction also increases 
instructor time both in preparation and delivery. Instructors need planning and implementation time to increase 
student satisfaction concerning interaction in the online classroom. Additional studies could provide more data on 
just how much time is needed and how instructors can be supported to focus on increasing student interaction and 
satisfaction. 
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How to Teach Students to Learn: Techniques Used to Increase Student 

Capacity to Self-Direct Learning 

Gregory M. Francom - The University of Georgia 

An important aim for educational institutions is to increase learner self-direction among students (Bolhuis, 
2003; Guglielmino, 2008; Meichenbaum & Biemiller, 1998). Learner self-direction is defined as the ability to self- 
direct and self-regulate one’s own independent learning processes (see Brockett & Hiemstra, 1991; Candy, 1991). 
Graduates will need self-directed and self-regulated learning (SRL) skills to help them keep up in career, personal 
and academic settings (Guglielmino, 2008; Zimmerman, 1994). In the United States and other similar countries, 
graduates’ will enter an economy characterized by high career mobility (U. S. Bureau of Labor Statistics, 2008), and 
abundant information (Lyman & Varian, 2003), making learner self-direction both more possible and more 
necessary than it has been in the past. Y et concerns have been raised nationally in the United States about the ability 
of higher education institutions to prepare students with the skills necessary to learn independently and keep up with 
career changes (The Secretary of Education’s Commission on the Future of Higher Education, 2006; The Secretary's 
Commission on Achieving Necessary Skills, 1991). In addition, preliminary studies at two different universities in 
the United States have found no significant increase in SDL skills among students from freshman to senior years 
using scales based in the SDL literature (Litzinger, Wise, Lee, & Bjorklund, 2003; Preczewski, 1997). 

U.S. higher education institutions have been challenged to develop new and innovative pedagogies to 
support previously neglected skills such as those needed for independent learning (The Secretary of Education’s 
Commission on the Future of Higher Education, 2006; The Secretary's Commission on Achieving Necessary Skills, 
1991). These types of outcomes have been advocated more particularly in science learning on the national stage by 
the National Science Education Standards of the National Research Counc il (1996). In this report, teachers are 
encouraged to foster learner self-direction by offering students the opportunity to take responsibility for personal 
learning, conduct self-assessments, and participate in the design of learning environments (see also Bransford & 
Donovan, 2005). 

Similarly, institutions and governing bodies outside of the United States have long emphasized the 
importance of SDL and lifelong learning for education. Notable initiatives and educational implementations of SDL 
related activities have been implemented in European countries such as Belgium, The Netherlands, Switzerland, 
France, Italy, Greece and the United Kingdom with national reports focused on SDL from many of these countries 
(Straka, 1997). SDL-focused reports of research and advocacy have come from areas as diverse as Asia (Chu & 

Tsai, 2009; Mok et al., 2007), Australia (Candy, 2004), and South Africa (Lindh & Hugo, 2005). It is clear that 
many countries around the world believe that fostering SDL skills is vital in our global economy. 

Partly in response to such mandates and challenges, higher education institutions in the United States and 
elsewhere have increasingly adopted language in mission statements indicating the importance of helping students 
self-direct, learn independently or become lifelong learners (Guglielmino, 2008). However, there is not wide 
agreement on how to best foster learner self-direction among students. 

Much of the SDL literature from adult learning views learners as universally highly self-directed during 
learning activities; needing only the opportunity to take charge of their own learning. These highly self-directed 
individuals who initiate their own learning are hypothesized to learn more deeply and meaningfully (Knowles, 1975, 
1980). Consequently, SDL models have advocated removing support and guidance for learning, placing the 
responsibility for learning squarely in the learner’s hands (e.g. Hammond & Collins, 1991; Knowles, 1975). Such 
approaches may be appropriate if certain assumptions about learners are met (Knowles, Holton, & Swanson, 1998). 
However, experience indicates not all adults are highly self-directing nor is self-directed learning always the best 
learning solution (Brockett, 1994). In formal education, for instance, learners may not be considered highly self- 
directed adults, and contextual factors such as curriculum requirements and standards usually affect the provision of 
SDL activities (Garrison, 1997). Students in formal education have been found to preliminarily need support and 
guidance for learning in the form of teacher-directed activities (Howland & Moore, 2002; Raidal & Volet, 2009). 
Thus, fostering learner self-direction in formal education involves more than simply reducing the amount of support 
and guidance given to learners (Brockett & Hiemstra, 1991; Candy, 1991; Merriam, Caffarella, & Baumgartner, 
2007). Approaches to learning and instruction that simply reduce or remove learning support may also be highly 
detrimental to the acquisition of knowledge in long term memory (Kirschner, Sweller, & Clark, 2006). 

Consequently, some guidance and support for learning may be needed with the aim of helping learners become more 
self-directed over time. In addition, the shift from teaching only for subject matter acquisition toward higher order 
skills such as learner self-direction is a formidable challenge for many teachers (Bolhuis, 2003). There is a need for 
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realistic guidance on how to foster learner self-direction within the contextual constraints of formal educational 
settings and among students who may not be universally highly self-directed. 

To meet this aim, this paper synthesizes literature from SDL and SRL for principles for fostering learner 
self-direction in formal education and describes a learning environment based on these principles. The learning 
environment is illustrated using examples from science learning. The definition of SDL in this paper encompasses 
SRL elements (see Loyens, Magda, & Rikers, 2008; Pilling-Cormick & Garrison, 2007). 

Principles for Fostering Learner Self-Direction 

Four main prescriptive principles for fostering learner self-direction in formal education can be extracted 
from SDL theory, models and research; 

• Match the level of self-directed learning required to learner readiness 

• Progress from teacher to learner direction of learning over time 

• Support the acquisition of subject matter knowledge and learner self-direction together 

• Flave learners practice self-directed learning in the context of learning tasks 

These principles will be discussed and supported with relevant theoretical and empirical literature. 

The first principle for fostering learner self-direction involves matching the level of SDL required in 
learning activities to learner readiness (Bolhuis, 2003; Brockett & Hiemstra, 1991; Grow, 1991; Flammond & 
Collins, 1991; Knowles, 1975). Adult learning literature acknowledges that individuals may be highly self-directing 
in some situations and not in others, or that they may be somewhere in-between high and low on the self-directing 
continuum (Candy, 1991; Knowles, 1980; Knowles et ah, 1998). A learner’s level of relevant domain knowledge 
and self-directed learning experience will affect whether they are able to self-direct learning in a given situation 
(Candy, 1991; Grow, 1991). SDL activities that this principle applies to may include allowing learners to set 
learning goals, specify what will be learned, determine the pace of learning, and evaluate learning outcomes 
(Fliemstra, 1994). If a learner is required to do these activities without regard to readiness, he or she may fail to learn 
or increase in learner self-direction (Brockett & Fliemstra, 1991; Candy, 1991). 

Findings from studies of SDL and SRL indicate that many learners are not ready to completely control a 
learning situation, and may need to first experience teacher-directed learning (Flowland & Moore, 2002; Raidal & 
Volet, 2009). Dynan, Cate and Rhee (2008) found that students whose SDL readiness is matched to a requisite 
learning structure (structured learning or unstructured learning) increased in self-direction over the course of a 
semester to a greater degree than those who were not matched. Bhat, Rajashekar and Kamath (2007) found that a 
high level of SDL activities helped high performing students learn while these activities did not benefit lower 
performing students. These findings suggest that matching the level of SDL activities required to learner readiness 
may be important for helping students learn and increase self-direction. 

The second principle advocates progressing from teacher to learner direction of learning over time 
(Bolhuis, 2003; Brockett & Fliemstra, 1991; Candy, 1991; Grow, 1991; Meichenbaum & Biemiller, 1998). This 
principle takes a learner from his or her current level (as suggested in principle one) toward higher self-direction 
over time. Prescriptive models of SDL offer practices for gradually increasing learner direction of the learning 
process. For example learners can be increasingly allowed to set learning goals, specify what will be learned, choose 
learning resources and evaluate learning outcomes as a learning experience progresses (Grow, 1991; Fliemstra, 
1994). Such an approach should take into account the first principle and provide learners SDL activities that match 
their readiness. Learners’ abilities to self-direct may increase as opportunities to self-direct learning are increasingly 
provided (Bolhuis, 2003; Grow, 1991). 

Azevedo, Cromley and Seibert (2004) suggest that shifting learning responsibility toward learners over 
time by scaffolding and fading support for SRL and SDL skills is vital for the teaching of self-direction. Fladwin, 
Wozney and Pontin (2005) followed the SRL development of graduate students in a research methods class for 
several months. They found that the general process in this setting involved teacher-direction that progressed to co- 
direction and finally student-direction of the learning process. This shift toward student direction of the learning 
process was also found in elementary school classrooms considered high in SRL (Perry, VandeKamp, Mercer, & 
Nordby, 2002). Schunk and Rice (1993) found that fading self-regulatory instructions were superior to self- 
regulatory instructions that were not faded in helping students with reading problems to self-regulate their learning. 
These studies suggest that gradually increasing learner direction of the learning process may foster learner self- 
direction. 
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The third principle for fostering learner self-direction involves supporting the acquisition of subject matter 
knowledge along with learner self-direction. Cognitive strategies (such as those required for SRL and SDL) are 
thought to require the use of intellectual skills (concepts, rules, etc. of a discipline) which require basic knowledge 
of subject matter (Gagne, 1985). Theoretical models of SDL have recognized that some domain knowledge is 
necessary for learners to be able to take responsibility for learning (Bolhuis, 2003; Grow, 1991). Learners should be 
introduced to relevant domain knowledge including underlying principles, procedures for knowledge acquisition, 
and generalizability of knowledge and practices, as they practice self-direction (Bolhuis, 2003; Vermunt & 
Verschaffel, 2000). 

Extensive domain knowledge may enable learners to free up working memory for processes related to self- 
regulation and self-direction of learning (Sweller, Van Merrienboer, & Paas, 1998). Experts have elaborate mental 
knowledge structures that enable them to monitor progress, choose appropriate strategies, and decide on appropriate 
solutions to problems (Chi, 2006). Within the self-regulated learning literature, Glaser and Brunstein (2007) found 
that providing instruction on both subject matter and SRL skills was more effective for helping students to control 
their learning than simply teaching subject matter. Cotterall and Murray (2009) provided SDL opportunities to 
students in a language learning course including allowing students to decide what to learn and choose resources to 
use for learning. The authors conclude that these elements of the course structure along with subject matter 
acquisition contributed to an increase in learner self-direction. 

The fourth principle for fostering learner self-direction advocates practicing SDL in the context of learning 
tasks. Studies of self-directed learners describe these learners as task-oriented, with the practical aim of applying 
learning to a specific task (Houle, 1961; Tough, 1979). Consequently, models of SDL have advocated providing 
learning that is centered on tasks that learners are likely to encounter in the future (Bolhuis, 2003; Hammond & 
Collins, 1991). As learners engage in these tasks, they may be required to do such SDL activities as: (a) choosing a 
learning path; (b) finding, evaluating, and applying information to complete tasks and solve problems (Bolhuis, 
2003; Candy, 1991); (c) monitoring and adjusting personal learning as needed (Butler & Winne, 1995; Garrison, 
1997) and; (d) determining ways in which personal performance should be improved (Bolhuis, 2003; van 
Merrienboer & Sluijsmans, 2009). Practicing SDL in the context of tasks may foster learner self-direction while 
increasing the relevance and usefulness of learning activities. 

Conceptual connections have been made between regulation of one’s own learning processes and problem- 
solving and experimentation activities (Mayer, 1998; Winne, 1997). For instance, Winne (1997) hypothesizes that 
students will “bootstrap” self-regulated learning skills from experiences in which they are engaged in goal-directed 
learning tasks. Connections have also been made in the literature from the implementation of problem-based and 
task-centered learning to self-direction (Gurses, Acikyildiz, Dogar, & Sozbilir, 2007; Hung, Jonassen, & Liu, 2008; 
van Merrienboer & Kester, 2008; Stewart, 2007). Woods (1996) found that student self-perception of SDL ability 
increased over time in a problem-based learning environment. Blumberg and Michael (1992) implemented a 
partially teacher directed and problem-based curriculum and observed increases in students’ SDL activities as 
measured by self-reports, program evaluations and library circulation data. Sungur and Tekkaya (2006) found that a 
problem-based learning environment enhances SRL skills more than a traditional lecture based environment. 
Findings from these studies support the hypothesis that centering learning on realistic tasks or problems provides an 
environment in which learner self-direction can increase. 

A Learning Environment Designed to Foster Learner Self-Direction 

Four principles for fostering learner self-direction have been presented; 

• Match the level of self-directed learning required to learner readiness 

• Progress from teacher to learner direction of learning over time 

• Support the acquisition of subject matter knowledge and learner self-direction together 

• Have learners practice self-directed learning in the context of learning tasks 

These principles have implications for the design of learning environments and can be implemented in a 
variety of settings. In this section, a description of what a learning environment focused on fostering learner self- 
direction might look like is given using examples from science learning in higher education. The practices presented 
in this paper are based on SDL research and theory (e.g. Bolhuis, 2003; Brockett & Hiemstra, 1991; Garrison, 1997; 
Knowles, 1975; Tough, 1979), and task-centered models of instruction (Collins, Brown, & Newman, 1989; van 
Merrienboer & Kirschner, 2007; Merrill, 2002a). 

A student in a learning environment designed to foster self-direction does tasks that represent activities 
done in the professional world outside of school (principle four) and follows a process similar to those described in 
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studies of the SDL experience (e.g. Brockett & Hiemstra, 1991; Knowles, 1975; Spear & Mocker, 1984; Tough, 
1979). The SDL process can be described as having four phases; initiation, acquisition, performance and assessment 
(see figure 1). 

The overall approach to fostering learner self-direction involves repeatedly showing students a learning 
task, presenting core concepts that are relevant to the task, modeling how to apply the core concepts to the task, 
having students perform the task, and assessing student performance of the task. The process repeats within the class 
in task sequences, and over time, responsibility for initiation, acquisition, performance and assessment is shifted 
from teacher to student. 




Figure 1. Phases of the self-directed learning process that occur within a learning task. Students initiate a learning 
task, acquire knowledge relevant to the task, perform the task and receive feedback on their task performance. The 
responsibility for these activities is shifted toward students over time. 

The Initiation Phase - Introducing Students to the Learning Task 

At the start of a unit, students are introduced to a challenging learning task (principle four; see table 1). 
Learning tasks are activities that students do that require them to apply knowledge from the course content to 
perform activities (cognitively and/or physically) that require the use of skills similar to those used in professional 
practice outside of school. Learning tasks should be based on the activities that students are expected to do with 
knowledge and skills after they finish the class (see the "determining learning tasks" section below; van Merrienboer 
& Kirschner, 2007; Merrill, 2002a). In the initiation phase of learning, students also make preliminary plans for how 
to accomplish the learning task and decide on a general plan to apply information to the task (Butler & Winne, 

1995). At this stage in the process, however, students likely lack the required knowledge and experience to complete 
the learning task. This approach may help motivate students to mindfully process course materials in search of 
information relevant to the learning task (e.g. Piaget, 1977). In addition to introducing the task, a teacher should also 
activate students’ prior knowledge by having them relate what they already know to the learning task (Merrill, 
2002a). 

As students become more knowledgeable and self-directed, they are given more opportunities to choose 
tasks to learn and perform. Students are also increasingly required to choose appropriate learning resources and plan 
how to approach learning tasks over time (principle two; see table 2). 

An example of how the initiation and other phases of the learning processes are implemented can be 
provided from a learning task within a biology class genetics unit. The Learning task is centered on a case study 
called, “Those old Kentucky Blues (Leander & Huskey, 2008),’’ that requires students to determine the origin of a 
patient’s blue skin using genetics knowledge. The teacher introduces the learning task to students in a presentation 
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that sets up the case, explaining that a clinic patient with blue skin presents herself and that students must determine 
the source of her skin discoloration. The teacher also assigns students to read a short paper explaining the problem. 
As students first learn about the task, they lack the genetics knowledge necessary to complete it, but they begin to 
determine what knowledge and skills they lack and think about how to approach the task. In this example, the 
teacher has chosen “Those Old Kentucky Blues” as a learning task for students to do, however, if students are ready 
for more SDL opportunities, a teacher can allow students to choose the tasks they wish to be engaged in (see table 
2 ). ' ' 

The Acquisition Phase - Acquiring Domain Knowledge and Modeling Application 

In the learning phase, students acquire the knowledge that they need to perform the learning task (principle 
three). Students leam two main types of knowledge in the learning phase, knowledge from the core concepts of the 
discipline, and knowledge about how to apply core concepts to the learning task. A teacher can use a variety of 
techniques to present core concept information including lectures, textbook readings, online presentations, etc. 
Learning how to apply knowledge of core concepts to a learning task is best done through teacher modeling and 
repeated student experience (Bandura, 1991; Collins et al., 1989). Student experience is provided in the performance 
phase, while modeling is provided in the current (learning) phase. A teacher models a learning task by showing 
students how to do it while calling attention to important performance aspects, discussing reasons why the task is 
done in a certain way and explaining the thought processes that are needed to complete it (Collins et al., 1989). 
Modeling particularly supports SDL skills when the model makes some mistakes and later corrects these mistakes 
(Kitsantas, Zimmerman, & Cleary, 2000). 

Table 1. 

Teacher and student activities in a learning experience designed according to the four principles for fostering 
learner self-direction. 



Learning What a teacher does What students do Cognitive activities 

Phase important for SDL/SRL 



Initiation 


• 


Present a new 


• 


Make learning task 


• 


Motivation 






learning task 




plans 


• 


Activation of prior 




• 


Activate student 


• 


Choose learning 




knowledge 






prior knowledge 




resources 


• 


Goal-setting 












• 


Strategic planning 


Learning 


• 


Present core 


• 


Actively determine 


• 


Subject matter 






concepts that are 




relevant aspects of 




acquisition 






relevant to the 




information for task 


• 


Strategy knowledge 






learning task 




completion 




acquisition 




• 


Model application of 


• 


Learn strategies that 










core concepts to the 




could be used for 










learning tasks 




task completion 






Performance 


• 


Provide coaching 


• 


Final selection of 


• 


Self-management 






and feedback as 




task strategy 


• 


Self-monitoring 






needed 


• 


Perform/complete 


• 


Motivation 










the task 


• 


Induction 








• 


Use relevant domain 














knowledge 







Assessment 


• Provide task- 


• Assess a peers task 


• Self-reflection 




oriented feedback on 


performance 


• Self-evaluation and 




task performance 


• Self-assess personal 


assessment 






task performance 
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Following principle two, student acquisition of core concepts is supported through more teacher-directed 
methods at the beginning of the learning process. Modeling showing how to apply core concepts is also provided in 
teacher-directed ways. Flowever, teacher-directed instruction is faded over time, giving students an opportunity to 
carry out and adjust their information gathering processes with increasing independence. Highly self-directed 
students may be required to find, study and apply all knowledge on their own and complete learning tasks without a 
model (see table 2). 

In the “Those Old Kentucky Blues'’ learning task, after being introduced to the problem, students begin to 
learn about relevant topics within the biology field such as dominant, recessive, co-dominant, and sex-linked genes, 
and other topics. Students leam about these topics through lectures, textbook readings, media presentations and SDL 
activities. As students learn the material, they actively search for ways that the information can be used to determine 
why a person might have blue skin. At the beginning of the genetics unit a teacher introduces the major concepts of 
genetics to students through more teacher-directed methods (including lecture, one on one instruction and readings). 
Later, students are required to find some or all of their own learning resources to complete genetics learning tasks 
(e.g. by searching through the textbook or looking online). When students have finished learning about core 
concepts from genetics, the teacher takes students through the process of applying these concepts to the learning 
task. To model the process, the teacher takes on the role of a coping model, making the mistake of applying an 
incorrect inheritance pattern and realizing that, based on family tree information, this pattern does not fully account 
for the blue skin. Next, the teacher explains his or her thinking process for applying the correct inheritance pattern 
while pointing out how and why this pattern applies to the problem. The teacher does not completely solve the 
problem, requiring students to finish solving it independently. Later in the genetics unit, students must complete 
learning tasks with minimal modeling and must determine how best to use relevant biology concepts to do this. 

The Performance Phase - Applying Knowledge and Completing the Learning Task 

In the performance phase, students apply their recently learned knowledge to complete the learning task. In 
a higher education science classroom, students can complete a learning task as part of a class assignment and then 
provide evidence that they completed it. This evidence might take the form of a written paragraph, an artifact 
resulting from task completion, or performance of the task with the teacher watching. Along with evidence of task 
completion, students should be required to explain what strategies were used to complete it and why. Learning task 
performance requires students to practice vital SDL skills including self-monitoring, self-management and 
motivation to stay engaged in a task (Garrison, 1997; Zimmerman, 2002). In self-monitoring, students observe, 
judge and react to the learning task, adjusting learning strategies that are not working (Butler & Winne, 1995; 
Zimmerman, 1998). Self-management refers to student management of the task performance process as they take the 
steps necessary to complete the task (Garrison, 1997). Early in the learning experience a teacher provides coaching 
and feedback to help students self-monitor, self-manage and stay motivated. Later in the learning experience, or if 
students are highly self-directed, support for task completion is reduced (principle two). 

In the “Those Old Kentucky Blues" learning task, students are required to determine the genetic source of 
the blue skin after learning about how traits are inherited (core concepts) and being shown how inheritance patterns 
might be applied to the problem. In this case students must hand in a short paper indicating what the source of the 
blue skin might be, how they came to this solution, what strategies they used and what genetics knowledge was 
relevant to the problem. If students need help while working on the learning task, they contact the teacher, who 
provides guidance and feedback. During the task completion process, students learn how to apply genetics concepts 
effectively as they discover the ways genetics knowledge can be used. In addition, students gain SDL experience 
selecting learning strategies to complete tasks. Later in the genetics unit, students must independently complete 
other tasks without teacher help. They will continue to use knowledge of inherited traits and more advanced genetics 
concepts to do this. 

The Assessment Phase - Assessing Learning Task Performance 

After the learning task has been completed, the teacher gives a grade and feedback telling how well 
students did. The teacher also indicates which aspects of students’ task performance were high and low quality and 
why (Shute, 2008). Students are also required to assess a peer’s or group’s work, providing similar feedback. Finally 
students might be required to self-assess their own work indicating strengths and weaknesses of personal task 
completion. Students who concentrate on specific aspects of task performance and self-assess may continually 
improve personal performance efforts (Butler & Winne, 1995; Ericsson, Krampe, & Tesch-Romer, 1993). For 
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highly self-directed students, a teacher can require students to self-assess learning task performance through a 
comparison to course criteria or a peer’s performance (Garrison, 1997; Zimmerman, 2002). 

In the “Those Old Kentucky Blues” learning task, students receive informative feedback from a teacher 
assessment, peer-assessment and self-assessment. Students assess a peer’s (or peer group’s) write-up after handing 
in their own using a rubric provided by the teacher. Later in the learning experience students may also be required to 
self-assess their task completion, using a similar rubric. In this problem, there is not necessarily one correct answer, 
so grading does not focus on whether the task solution was right or wrong. Through reflection on peer and personal 
performance, students gain a greater understanding of how genetics knowledge can be applied and transferred to 
different situations (van Merrienboer, 1997; Merrill, 2002a). 

Table 2. 

Teacher-directed, moderate and student-directed approaches to learning activities within each phase of the learning 
process. 



Course element 


Teacher-directed approach 


Moderate approach 


Student-directed approach 


Learning task choice 
(initiation phase) 


Teacher chooses and 
provides all learning tasks 


Teacher provides a list of 
learning tasks for students 
to choose from 


Students choose learning 
tasks to engage in 


Learning resources for 
core concepts 
(Learning phase) 


Teacher provides all 
learning resources (lecture, 
textbook, online etc.) 


Teacher provides some 
learning resources and asks 
students to find and apply 
additional resources 


Students find, evaluate and 
apply learning resources 
independently 


Modeling application 
of core concepts to the 
learning task 
(Learning phase) 


Teacher provides extensive 
and specific modeling 
showing how to apply core 
concepts to learning tasks 


Teacher only models a part 
of the application process 


Students apply core 
concepts independently 
without a model 


Learning task 
performance 
(Performance phase) 


Teacher provides extensive 
coaching and feedback for 
task performance 


Teacher provides coaching 
and feedback when 
students request it 


Students provide own 
coaching and feedback 


Assessment of 
learning (Assessment 
phase) 


Teacher assesses student 
performance 


Students assess personal 
and peer performance and 
a teacher also provides 
assessment 


Students assess personal 
and peer performance 



Note. Ideally, a learning experience gradually shifts from teacher-directed to student-directed approaches over time. 

This process of initiation, acquisition, performance and assessment repeats in the genetics unit and 
continues in other units of the class. Repeated tasks that require students to apply similar information help students 
develop a conceptual knowledge of the principles involved (e.g. genetics) and transfer this knowledge to other 
situations when needed (van Merrienboer, 1997). SDL skills are fostered in this process through learning tasks that 
require students to apply knowledge and through a gradual shift of responsibility for learning toward students. By 
the end of the class students are required to direct many aspects of their own learning processes (see table 2). They 
must choose learning resources, plan how to complete learning tasks, monitor and adjust their task performance, and 
self-assess their learning. 

The Design of a Learning Environment to Support Learner Self-Direction 

The process above involves students in learning tasks and shifts responsibility for learning toward students 
over time. The creation of this learning environment takes some careful planning, design and implementation 
activities. General guidelines for the design of the learning environment described above will now be given 
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including guidelines for determining learning tasks, finding out about students, designing the learning experience, 
developing course materials, and adjusting the learning experience. 



Determining Learning Tasks 

The question, “what should student’s learn?” usually guides a teacher/designer in making curriculum 
decisions. Within formal education, the subject domain is usually specified (e.g. biology) along with specific units 
of items that should be learned (e.g. genetics, evolution, ecology). In contrast, task-centered models and SDL theory 
suggest that what students learn should be based on a task analysis, or an analysis of what students will likely be 
required to do after they are finished with the learning experience (Hammond & Collins, 1991; Knowles, 1980; van 
Merrienboer & Kirschner, 2007; Merrill, 2002a). This changes the question from, “what should students learn,” to, 
“what will students do after this class is over?” This thinking is consistent with the fourth principle, centering 
learning on tasks. These learning tasks are based on the activities that students are expected to do once they are 
finished with a class. For example, in a writing class, students should do writing tasks that are typically performed in 
business situations (i.e. writing a proposal for action, communicating clearly by email, creating meeting agendas) if 
students are likely to seek out a career in business. 

Finding out about Students 

The above example also implies that a teacher/designer needs to gather information about students before 
class. Typical student aspects that are important to the design of learning experiences include how much students 
already know about a subject, what motivation students have to learn, and what professional activities students are 
likely to do in the future (Dick, Carey, & Carey, 2005; Gagne, Wager, Golas, & Keller, 2005). Principle one 
indicates that teachers must also know how ready students are to self-direct their own learning. A teacher/designer 
can find this out through a class discussion, an instrument designed to measure SDL or SRL readiness (e.g. 
Guglielmino, 1977; Pintrich, Smith, Garcia, & McKeachie, 1993; Stockdale, 2003), a questionnaire, or other 
pretests. At the very least, a teacher/designer should estimate the SDL readiness of students using past teaching 
experience as a guide. Knowing about students’ readiness to take charge of their own learning activities can guide a 
teacher/designer in determining the level of control to give to students during the learning experience. For instance, 
if in the past students have shown low levels of successful engagement in SDL activities, the teacher should start 
with a lot of teacher control and then gradually move toward student control of the learning process (principle two). 

A teacher may also benefit from finding out what students want to learn and adjusting course items based 
on this input (Hammond & Collins, 1991). This approach to education has been advocated by authors in adult 
learning and SDL theory for learners who are highly self-directed (Knowles, 1975, 1980; Knowles et ah, 1998), but 
may be very difficult to implement when course content is tightly controlled or when students do not know what 
they want to learn. 

Designing the Learning Experience 

The design of the learning experience should start with the identification and ordering of learning tasks 
(principle four; van Merrienboer & Kirschner, 2007; Merrill, 2002b, 2007). These tasks should be ordered from easy 
to difficult and provide a conquerable challenge for students. In classes where the subject matter is already specified 
and controlled, a teacher will have little room to change what is taught. In these cases, however, it is still possible to 
use tasks. For instance, a basic course in electrical engineering may have required units such as electrical circuits, 
motors, generators, and computer systems. A teacher/designer can create a sequence of tasks for students to do 
within each of these areas based on professional engineering practice. For example, a teacher/designer can design 
two or three learning tasks that will require students to apply their knowledge of electrical circuits for the electrical 
circuits unit. An approach to the sequencing of tasks that moves toward student direction of the learning process 
(principle two) might involve creating a list of learning tasks for students to choose from and providing advice on 
which tasks to choose (Kicken, Brand-Gruwel, & van Merrienboer, 2008; van Merrienboer & Sluijsmans, 2009). 
Ultimately students might be given the opportunity to independently choose all learning tasks (see table 2). 

When specifying learning tasks, a teacher/designer must also determine the types of knowledge students 
will need to have in order to complete the tasks (principle three). Two types of knowledge should be provided to 
students. The first is knowledge about the domain(s) in which the task resides (core concepts) that is relevant to the 
completion of a learning task. In the engineering example, this is knowledge about engineering concepts relevant to 
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electrical circuits including knowledge about the different types of circuits, Ohms law, Kirschoff s laws, how 
circuits are built, etc. According to principle two, this knowledge should be provided with more teacher-directed 
instruction at first and then gradually a teacher/designer should shift the responsibility of finding and evaluating this 
knowledge toward students (see table 2). 

The second type of knowledge shows students how to apply core concept knowledge to a learning task. In 
the electrical engineering example, this information might include how to apply laws of current to the creation of an 
analog electrical circuit. This information is best provided through modeling that shows students how to complete a 
learning task and explains the thought processes leading to actions (Collins et al., 1989). To increase student 
direction of the learning process over time (principle two), the responsibility for modeling and application of 
concept knowledge should be gradually shifted toward students. 

In addition to designing learning tasks and the knowledge needed to complete a learning task, a 
teacher/designer will also need to design assessment instruments. Most higher education courses require a final 
assessment and mid-term tests, but assessment for fostering learner self-direction is much more. For each task 
students perform, they should receive a grade and feedback indicating what was done well and what could have been 
done better (Shute, 2008). The design of assessments should also follow principle two and gradually shift from 
teacher to student assessment. Findings have indicated that, if implemented correctly, self-assessment and group 
assessment by students tend to have high validity when compared to teacher assessments (Cho, Schunn, & Wilson, 
2006; Kilic & Cakan, 2007; Ryan, Marshall, Porter, & Jia, 2007). 

In addition to the above task design activities, a teacher/designer needs to determine in what order and how 
tasks should be presented to students. Overall, learning tasks should be designed to go from easy to difficult or from 
high to low support (van Merrienboer & Kirschner, 2007; Merrill, 2002a). Students can also do learning tasks 
individually or in groups with differing levels of support (Merrill & Gilbert, 2008). The sequencing of tasks can 
support a gradual shift from teacher to student direction of the learning process (principle two). For instance a design 
can feature a high amount of guidance and modeling when students are asked to perform a task at the beginning of 
class and fade this guidance and modeling later. 

Developing Class Materials 

After analyzing aspects of the learning situation and designing a learning experience, a teacher/designer 
must develop materials, media and tasks for student use (Dick et al., 2005; Gagne et al., 2005). Development 
includes the creation of media for student involvement in the learning tasks and the final selection of learning 
materials from which students will learn. Development also involves making decisions about how to best provide 
guidance and feedback to students as they perform tasks. Different media that can be used to present knowledge 
include in-class presentations, textbook readings, videos, web pages etc. Students should be increasingly given the 
opportunity to choose learning resources to use over time. For example at the beginning of a class, a teacher might 
provide a list of resources to students and suggest that they also look up further resources as needed to complete a 
task. In later tasks, the teacher could suggest fewer resources for learning and ask students to find additional 
resources themselves. Care should be taken, however, to assure that students know how to identify reliable 
resources. 

Making Needed Adjustments to the Learning Experience 

Constant effort is needed to determine the effectiveness of the learning experience and to revise it to better 
support learning and self-direction. Information that can guide the revision efforts can include informal student 
feedback, assessment data, classroom observations etc. (Dick et al., 2005). A chance for revising and improving a 
class occurs after students have viewed each learning resource, completed each learning task, and interacted with 
other course elements. Information about student experiences in doing SDL activities and interacting with tasks can 
help guide efforts to improve a class. 

When the newly-redesigned class is “tried out” with students, some of the most important discoveries are 
made. Experience in class might indicate a needed adjustment in learning task difficulty, amount of student direction 
required, amount of support and guidance given, amount of resources students must look up, and level of self- 
assessment required of students. For instance, students may find that a learning task is too easy or too difficult to 
perform with the provided support. They may be unable to self-regulate and self-direct learning to the degree that is 
required in the class. Such issues are an indication that adjustments need to be made to the learning experience to 
increase its effectiveness. 
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Conclusion 



Higher education is a means by which students prepare for lifelong learning and career performance. 
Increasing students’ ability to learn on their own should be an important goal for higher education learning 
experiences (Bolhuis, 1996; Grow, 1991; Guglielmino, 2008). Learner self-direction continues to be necessitated by 
increases in career mobility rates (U. S. Bureau of Labor Statistics, 2008), in which graduates will have to use SDL 
skills to keep up (Guglielmino, 2008). This paper presented and supported four principles for fostering learner self- 
direction in formal education from the SDL and SRL literature. The four principles can guide the design of learning 
environments intended to foster learner self-direction. The learning environment presented in this paper involves 
students in four phases of the SDL process while completing tasks. Throughout this process, responsibility for key 
processes such as choosing a task, acquiring knowledge, applying knowledge, monitoring performance and 
assessing performance is shifted toward students. It is hoped that the model presented above presents salient 
guidance to educators and instructional designers for providing learning experiences that foster learner self- 
direction. 

Based on preliminary studies (e.g. Litzinger et ah, 2003; Preczewski, 1997), it may be that, at present, 
higher education is overly focused on creating “good students” who are able to acquire knowledge only in the most 
teacher-directed situations. Instead, we can prepare students to be “intelligent engineers” who know how and when 
to independently acquire more knowledge, and apply this knowledge to accomplish useful tasks. As we move 
forward, we will need to move away from creating “good students,” and toward creating “intelligent engineers.” 
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Introduction 

Systematic instructional design provides a crucial vehicle for designers and teachers to develop efficient 
and effective learning environments (Dick, Carey, & Carey, 2005). Formative program evaluation provides an 
opportunity for designers and teachers to modify previous instructional experiences, to foster enhancement of 
student learning and performance (Russ-Eft & Preskill, 2001). This paper will outline the program evaluation 
conducted on a systematically designed instructional unit for a large community college in the Northeastern US, 
located in upstate New York. 



Program Overview 

Medieval Philanthropy: A Bible in Glass and Stone, is an 80-minute prototype program, which includes the 
introduction of Romanesque style architecture and the progression of the physical aspects of medieval architecture, 
from the birth of Romanesque style churches through the introduction of the ‘new’ Gothic style architecture. The 
lesson also includes a focus on the influence of the period’s social concerns, the European climate type, new 
discoveries, and the introduction of new building technology. This rich background information adds breath and 
depth to the developmental analysis of medieval architecture. 

The goals of this program are twofold. They include the following: a) Students’ ability to identify the 
physical attributes of Romanesque and Gothic style churches, b) Participants should also be able to identify the 
social and climate-type factors, which led to the development of these two architectural styles. Early field test 
results attest to students’ realization of these goals, when using this instructional program. 

This program was specifically designed to educate and motivate an approximately 33-student class of one 
hundred or two hundred level undergraduate, general studies students. With modifications, it may be used for more 
advanced coursework or larger class sizes as well. The use of multimedia, student participation, and underscoring 
connections to modern life have been proven to help facilitate an engaging experience for adult learners of all ages 
in this program, during this field test. 



Instructional Objectives 

Medieval Philanthropy: A Bible in Glass and Stone is designed so that students can attain the following two 
objectives by the end of the program. 1) Given examples and non-examples of the periods’ social factors and 
geographic climate type, students will identify the factors that influenced the progression of architecture, from 
Roman Basilicas to the introduction of Gothic architecture, with 80% accuracy. 2) Given examples and non- 
examples of physical architectural elements and whole building illustrations, students will classify features from the 
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Romanesque time period with 80% accuracy. 3) Given examples and non-examples of physical architectural 
elements and whole building illustrations, students will classify features from the Gothic time period with 80% 
accuracy. 

Instructional Materials 

Medieval Philanthropy: A Bible in Glass and Stone includes all of the materials needed for instruction and 
assessment on the lesson content and objectives. These materials include: a student participation (Apprentice) 
manual with complete lesson content, learning activities, practice objective assessment items, a pretest, a posttest, an 
instructor guide, and 2 supporting CDs including both manuals, video clips, and presentation slides. 

Evaluation Method 

Participants 

A total of 42 general studies students from two different class sections participated in this prototype tryout. They 
each belonged to a one-hundred-level medieval history course, at Hudson Valley Community College, in New York 
State. The single course instructor reported the following demographics from his observations: The average age of 
student participants in the day class was approximately 19 years old. There were three Caucasian females and 16 
Caucasian males. The average age of student participants in the evening class was approximately 27 years old. 
However, there were also two participants in his/her 50s and one in his 70s. The evening section included one 
African American and 21 Caucasian students. The male to female ratio for this section was about equally split. 

Process 

One instructor taught the lesson to his own students using the procedures described in the Medieval Philanthropy: A 
Bible in Glass and Stone Instructor Guide. The teacher administered and scored the pretest and posttest. An 
optional creative writing activity was included during the evening section, but not implemented for the daytime 
section. The creative writing activity, which asked students to create a cartoon strip incorporating key lesson 
concepts, was met with some opposition during the evening section; many of the more mature students (aged 35 and 
older) chose not to participate in the activity. 

Data Sources and Collection 

To conserve class time the pretest only consisted of 10 multiple-choice items: three items for objective one, four 
items for objective two, and two items for objective three. The posttest consisted of 14 multiple-choice items: 5 
items for objective one, four items for objective two, and five items for objective three. The student attitude survey 
and teacher questionnaire were adapted from previously constructed instruments, by Educational Psychology 
Department faculty members, Dr. Wilhelmina Savenye, Dr. James Klein, and Dr. Howard Sullivan. The data 
collected by the pretest, posttest, student survey, and instructor questionnaire will be used to evaluate and revise the 
instructional program. Permission was granted by Arizona State University’s Internal Review Board (IRB) for data 
collection, analysis, and publication. 



Results 

Student Achievement 

The mean percentage scores of the entire field-test cohort on the pretest and posttest show that the overall mean 
scores were 41.5% on the pretest and 77.9% on the posttest. Students’ pretest scores were near the 33% chance 
level on objectives 1 and 2. Students’ pretest scores were 64.3% for objective 3. However, the limited sample 
number of two items for objective 3 may have distorted those results. The posttest scores were above 80% on 
objectives 2 and 3 but below this level (61.5%) on objective 1. There were no statistically significant differences 
between the students’ achievement in the day section vs. the evening section. The overall pretest mean for the day 
section was 40.9%, as compared with 42% for the evening section. The overall posttest mean for the day section 
was 77.8%, as compared with 78% for the evening section. 

Student Attitudes 

The majority of the program participants enjoyed this program, as can be seen from the positive ratings on the 
student satisfaction survey. 90% thought it was interesting. 78% or more thought it was clear and an all-around 
good instructional program. When asked what they liked best about the program, students reported: the content, the 
instructional style, and the visual aids. Complete survey results are shown in the appendix. 
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Teacher Attitudes 

The single tryout teacher generally liked the program design, as indicated by his responses on the instructor 
questionnaire; he gave positive responses about ease of program use, organization, and appeal. The major strengths 
were said to be the visual cues and well-formatted assessment questions. The instructor suggested adding more 
pictures and removing the creative writing comic and What do we know? What do we want to find out? How can 
we find what we want to leam? What did we learn? (KWHL) chart activities. 

Discussion/ Suggested Revisions 

Based upon the data collected during the prototype tryout, the following considerations should be made for 

revision: 

About 22% of the student participants noted a preference for a slower instructional pace in the qualitative 
comment area. However, several students’ comments (approximately 10%) suggested adding more material or 
depth to the content in the qualitative response area. The instructor also reported that he would like to see more 
example pictures included of churches and stained glass windows. These ideas really complement each other. 
Adding more pictures would make the pace feel slower since they would allow students to spend more time on each 
concept. Due to the volume of suggestions, this revision will be immediately implemented. 

The instructor suggested removing the KWHL chart due to observing confusion emanating from students’ 
behaviors, which may have led to the observed lack of student participation in this activity. Future versions will 
present it as an “optional activity” in the instructor guide. 

The instructor only had enough instructional time to present the comic activity during the evening section. 
He reported a complete lack of participation from all students approximately 35 years old or older during the comic 
strip optional activity. He noted that they wouldn’t even get up to go into groups on the instructor questionnaire. 

Due to this activity’s radical instructional style which many instructors are not familiar with and the lack of student 
participation, it will be removed from future versions of this program. 

Conclusion 

This instructional program was designed to provide an opportunity for non-major undergraduate medieval 
history students to connect with information generally thought of as inconsequential for many in modern society. 
After three years of implementation, the instructor still uses the program to cover these specific curricular 
objectives. The program successfully engages these students through allowing them to explore relationships 
between historical events and common architectural features. Through focusing on making relevant connections and 
applying modern teaching strategies, designers and instructors may successfully capture students’ attention, 
motivation, foster positive learning attitudes, and help learners succeed. 
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Appendix 

Responses for Student Satisfaction Survey* 

Directions: Circle the answer that best describes how you feel about each question. Use the following scale to help 
you: 





Agree 

Strongly 


Agree 


No 

Opinion 


Disagree 


Disagree 

Strongly 


1. 


The program objectives were clearly stated 


17% 


64% 


12% 


7% 


- 


2. 


The program was interesting 


19% 


71% 


10% 


- 


- 


3. 


The information presented was clear to me 


24% 


55% 


14% 


7% 


- 


4. 


The practice exercises aided my learning 


12% 


69% 


17% 


2% 


- 


5. 


The video aided my learning 


26% 


57% 


17% 


- 


- 


6. 


The presentation slides aided my learning 


26% 


64% 


10% 


5% 


- 


7. 


The assessment was a reasonable way to tell 
what I had learned 


14% 


60% 


21% 


5% 


- 


8. 


This was generally a good instructional program 


14% 


64% 


17% 


5% 


- 




Too Hard 


About Right 


Too Easy 


9. 


The program was 


5% 


88% 


7% 



10 . What did you like most about this program? (30 comments total) 

• The material or the content. (17) 

• The slides, or the video, or the visual aids. (6) 

• The humor. (2) 

• The instructional style. (2) 

• The connection made between then and now. (1) 

• The teacher. (1) 

• The Wu Tang Clan music. (1) 

11. What did you like least about this program? (25 comments total) 

• It was hard to follow, or it went too fast, or it was too brief, or it was too short. (12) 

• The video or the slides. (5) 

• The content. (3) 

• Nothing. (2) 

• I’m tired, or I didn’t get to build a church, or I don’t like me. (3) 

12 . Please take a moment to write any suggestions you may have for improving this instructional program. 

(10 comments total) 

• Slowdown. (3) 

• More about stained glass, or more information, or more depth. (3) 

• No suggestions. (1) 

• Do this for multiple topics and future discussion. (1) 

• More pictures of local churches’ interiors and exteriors. (1) 

• Student, arrived 35 minutes late: “I had no idea of what we were doing. No real instructions or 
objectives.” (1) 

*Adapted from Dr. Wilhelmina Savenye, Dr. James Klein, and Dr. Howard Sullivan, copyright 2007 
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Introduction 

While high quality instructional design tends to be economically rewarding for organizational adopters, it 
also requires a substantial capitol investment that small businesses often forgo. Such deployments require relatively 
robust corporate budgets, which are unrealistic for most modern small businesses. Further, Paradise and Mosley 
(2009) report that in past economic downturns, organizational managers have tended to reduce investments in 
learning budgets and fail to reinstate those lost resources once the economy had recovered. The current amount of 
limited resources devoted towards systematic analysis, design, development, implementation and evaluation, of 
human performance technology has begun to require instructional designers to reinvent their roles. A team of 
educational technologists used client coaching, scaffolding, instructional design mentoring, and open-source 
learning management software to meet the training needs of a client in the fitness industry. 

In the past, most training was provided to many of industries’ early novice learners by expert mentors. A 
glance at historical training solutions might remind us of the importance of apprenticeships, which were heavily 
used by small businesses in areas such as architecture, metal working, masonry, farming, cottage industries, 
mechanics, folk arts, studios, etc. This early on form of performance training combined with the gift of modern 
pedagogical scaffolding tools, coaching, and open-source learning management software can result in professional 
quality, state of the art instructional design. 

Coaching has been a popular performance improvement strategy which when implemented often has a 
positive impact on individual and organizational learning (Luecke, 2004). Through supporting the client’s search to 
firmly identify goals and take action to reach those goals, the authors guided a small business in the transformation 
of face-to-face instruction into a high-quality blended learning program at a fraction of the cost of traditional in- 
house instructional design. The following evaluation report outlines the results of this project. 

Program Overview 

The Christian yoga online instructor-training program was designed as an introductory, asynchronous, 
three-month or two-hundred hour, interactive online learning opportunity, which is housed in the open-source course 
management system, Moodle. The online training is organized into eight instructor-facilitated modules; 1) 
Orientation, 2) Christian yoga philosophy, 3) Nutrition and healthy living, 4) Marketing Christian yoga, 5) Yoga 
industry, 6) Anatomy and physiology, 7) Alignment and postures, and 8) Teaching methodology. 

The goals of the online instructional component of this program are to a) provide students with an 
opportunity to acquire the declarative knowledge needed to become a Christian yoga instructor, and b) begin 
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building a professional support network. An instructional design team facilitated the transition from a face-to-face 
curriculum into an online hybrid-learning environment. The team possessed collective expertise in program 
management, network administration, instructional design, online learning, and the health fitness industry. The 
online learning environment supported students’ acquisition of technical declarative knowledge, comprehension, and 
analysis, of philosophy, nutrition, entrepreneurship, and vocabulary. The physical postures and kinesthetic 
alignment were the focus of the subsequent two-week or eighty-hour face-to-face portion of the course. 

The target population is comprised of men and women who worship Christ and would like to develop a 
Christian yoga ministry. Program participants should have taken at least 5 yoga classes, possess strong English 
language skills, successfully completed at least an eighth grade education, as well as feel comfortable using a 
computer and the Internet. 

Each of the three online program instructors received online facilitator training and professional 
consultation. The human performance technology consulting group personnel provided all training and consultation 
during the pilot program. The online instructor responsibilities included planning for instruction, developing 
instructional materials, tracking individual participants’ progress, answering questions, providing support and 
motivation for students, as well as facilitating group discussions. 

A Moodle course-management administrator was also trained and supported by the human performance 
technology-consulting group during the pilot program. The administrator’s responsibilities included uploading 
instructional materials, assessments, setting up the Moodle environment, and populating the Moodle environment. 
Formation of all milestone goals, expectations, an applied introduction to instructional design aligned with related 
content examples, work process, approval procedures, implementation, and evaluation plans, were planned 
collaboratively between the consulting group and clients. Training sessions and consultations for the instructor and 
administrator clients were provided face-to-face, via the phone, and through email correspondence for the duration 
of the pilot program. 



Evaluation Method 

Participants 

A total of 16 students from across the U.S. participated in this online pilot program. Some students had 
experience with online learning environments and some did not. All students voluntarily registered for the online 
instructor-training course. The average age of this predominately Caucasian student participant group was 
approximately 33 years old. All students in this pilot group were female. 

Process 



Three instructors and one administrator facilitated the delivery of the Christian yoga instructor training 
online learning program during a three-month period. A 90-minute face-to-face evaluation session was also 
conducted with the same 16 online students at the Christian yoga studio, during the 10-day face-to-face portion of 
the training program. 

Data Sources and Collection 

Three of the eight online modules, Christian yoga philosophy. Anatomy and physiology, and Alignment 
and postures, included several objective -type multiple-choice assessment items. The assessment items were grouped 
into several topic areas within each of their respective modules. All 16 online students were allowed to take each of 
the assessments as many times as needed, until each student answered 80% or more of the questions correctly. 

All eight online modules contained open-ended discussion prompts. The discussion prompts asked 
students to analyze the meaning they made of the materials provided. Students were also prompted to explore the 
applicability of the course content to various situations they may encounter as Christian yoga instructors, within a 
discussion board format. 

The ninety-minute face-to-face evaluation session was used to elicit feedback on the students’ experiences 
with the online learning program and explore desired features for a future online post-program graduate portal. The 
evaluation tools included a silent, anonymous, pencil-and-paper student satisfaction survey, three small group focus 
panels, and a large group discussion to verify data gathered in the small focus panels and explore an online post- 
program graduate portal. 

After the face-to-face student evaluation session, each of the three instructors also received an opportunity 
to complete an online survey used to evaluate their experiences with the online learning program development and 
the human performance technology-consulting group. Two instructors collaborated to respond to one survey as well 
as a third instructor who also completed an instructor satisfaction survey. 
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Results 



Student Achievement 

All 16 students were eventually able to successfully answer 80% of more of the objective-type multiple- 
choice questions asked about each topic assessed in the Christian yoga philosophy. Anatomy and physiology, and 
Alignment and postures, modules. During a small group focus panel discussion in the face-to-face evaluation 
session, students reported to the facilitator that many of them took advantage of the option to retake the quizzes and 
tests, as many times as they needed. However, some students voiced concerns over the wording of some of the 
questions during the small group focus panels. All 16 students were also able to respond completely to the 
discussion questions posed in each of the eight online modules, with facilitation and instructor prompts. However, 
some students voiced concerns about redundant topical discussion questions during the small group focus panels. 

Student Attitudes 
Satisfaction Survey Results 

All 16 students responded to the paper-and-pencil online student satisfaction survey. Both the quantitative 
and qualitative satisfaction survey data revealed that students generally enjoyed their learning experience. The 
overwhelming majority of students reported that the online course meet their expectations (4.6 rating out of 5.0) and 
rated the course content at 4.3 out of a 5.0-point scale. Students mostly reported enjoying the performance of each 
of the three course facilitators as evidenced by the 4.5 averaged rating out of a 5-point scale. Data collected also 
showed that students appreciated the instructors’ responsiveness to student questions (4.4 out of 5.0) and that the 
required reading documents were generally easily accessible from the online environment (4.25 out of 5.0). 

Students also reported the course as being generally well paced (4.6 out of 5.0) and well organized (4.7 out of 5.0). 
Students’ opinions tended to vary a little more in their opinions of the necessity of the RYT designation, as 
evidenced by the 4.1 rating out of the 5.0 scale of importance. 

Student’s qualitative survey feedback was generally positive. Many students remarked that they enjoyed 
their instructor’s supportive and positive online personalities, subject matter expertise, and their quick responses to 
content-related questions as well as their technology challenges. Many students also reported learning a great deal 
of new information, enjoying the online environment, and feeling connected with the learning community. Some 
students remarked that a future RYT designation would be relatively important to obtain but that it wasn’t something 
they felt they needed at this time, while others would have liked to complete the training with this designation. 

Face-to-Face Focus Group Results 

All 16 students also participated in one of three face-to-face, small focus groups (e.g. course 
content/pacing/workload, technology, student-teacher communication/fellowship). Each group was asked the same 
four questions; a) What should stay the same?, b) What should be added?, c) what should be taken away?, and d) 
What could be improved? After each small group had a chance to brainstorm and discuss their input, the 
information was reviewed for completeness and verified with the large 16-student group. Students were asked to 
comment on their experiences with the online portion of the course, however some students also contributed 
comments that were focused on the face-to-face classroom experience. 

Course content, packing, and workload focus group 

Students reported that they enjoyed the Christian part of the course as well as having a pacing calendar 
presenting an overview on the whole module. Many students recommended that a place for immediate, anonymous 
feedback be incorporated into each module. Students also requested a video chat feature to be added to the course 
and more instruction on the purpose and path within the spiritual basics section of the course. Students 
recommended that the redundancy within the spiritual basics and orientation modules be reduced. Many 
complained that they felt they were being asked the same discussion topics during these two modules. Students 
also suggested that all of the handouts in each module be compiled into one printer-friendly booklet, introducing 
more yoga sooner, providing more explicit instructions on how to use some of the Moodle features such as the 
instant chat and discussion boards. Several students requested additional organization, content alignment, and test 
preparation to be added into the course. 

Technology focus group 

Students reported that they enjoyed the participant information profiles, pictures, discussion forums, the 
ability to review wrong answers, as well as the ability to access and download handouts even after each module 
ended. Again students requested more instructions on how to use different features within Moodle, as well as more 
instructional DVDs on postures and anatomy. Students also reported a desire for the ability to provide status 
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updates, such as those available in Face Book. Students also discussed their dismay with the enrollment problems 
from modules 4, 5, and 6. Again, students requested access to additional communication features. 

Student interaction, teacher interaction, and fellowship focus group 

Students generally reported that they liked the timing of interactions, getting to know each other, having 
access to everyone’s different points of view, and receiving personal emails/responses from their teachers. Some 
students reported feeling isolated towards the end of class, requesting more discussion forum opportunities. 

Students in this focus group also requested a place for immediate, anonymous feedback and other communication 
features/instructions such as a synchronous chat. Students requested more detailed information on each teachers’ 
role(s), more direction, clarity in the orientation, and video explanations. 

Instructor Attitudes 

All three pilot course instructors participated in an online instructor satisfaction survey, which was 
distributed following the face-to-face portion of the school’s first hybrid program student cohort. Two of the three 
survey participants completed one online evaluation form collaboratively. Both instances of the survey results were 
kept anonymous by one of the evaluation tool’s software features. Each of the three instructors was brand new to 
online educational environments. However, they had been conducting face-to-face Christian Yoga Instructor 
Training courses for approximately three years prior to when this instructional design project began. 

In general, the survey results revealed that the three instructors were satisfied with their instructional design 
project work, course implementation, and facilitation. This is evidenced by the five point Lykert scale measurement 
tool, which was created for this unique project with input gathered from all stakeholders. For example, when the 
instructors were asked to rate their feelings about the within-module organization, overall course organization, 
course pace (or rhythm), overall course quality, the instructional design team’s services, the instructional design 
process, and the Moodle learning environment, the instructor’s measured their satisfaction at the satisfied or very 
satisfied level, 93% of the time. 

Overall, the qualitative evaluation questions also revealed a satisfied general consensus about the project. 
One instructor remarked that, “The course content brought our education to the 'Gold Standard' outcome we so 
desired. It made us think long and hard about what we are teaching and the best way to do it.” Another instructor 
remarked that she, “Learned a whole new set of skills for myself, and enjoyed the teamwork.” Several other 
comments were also made about the talents held by the human performance technology consulting team. 

The few concerns noted in the evaluation results centered mostly on the early portion of the instructional 
technology design learning curve and becoming acquainted with the learning environment. Survey respondents 
shared their apprehensive feelings about the amount of new information being presented, the level of content 
preparation expected from the teachers, not being able to get into the Moodle environment earlier in the project for 
help with conceptualization, some early confusion with the technology, the online portion of the program’s pacing, 
and the quality of student posts. Several of these concerns have already been resolved with the hands-on experience 
gained by the Christian yoga instructor team through the pilot program. 

Suggested Revisions 

The majority of the feedback gathered from both students and the instructors has been positive. However, 
valuable life experience and the field of evaluation often tell us that there is usually room for improvement and 
growth. It should be noted that due to the one-time case-study nature of this project, as well as both small student 
and instructor sample sizes, change decisions should be carefully calculated and implemented. In addition, the 
development of a continued formative evaluation plan is advised as well as further data collection and analysis. 

Based upon the data gathered from this pilot hybrid training program’s online component evaluation, 
development opportunities have been classified into four different areas; course structure, content, assessment tools, 
and Internet learning environment. 

Course Structure 

Suggested adaptations within the course structure focus on communication and organization. Some of 
those who participated in this evaluation felt uncomfortable with the pacing, at different times during the online 
portion of the course. Extra effort should be made to communicate pacing and work expectations with students. 

The calendar feature within the Moodle course should be provided, updated, and monitored by the current instructor 
facilitator or administrator. Also, the supporting reading handouts or materials could be compiled into one 
document for each module. Access to additional communication options within Moodle, such as instant chat seems 
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to be important for students and warrants investigation. More opportunities for community building were requested 
in the form of continued forum discussions throughout the later portion of the online program. 

Content 

Due to several student comments made about the redundancy of discussion forum topics and responses, a 
detailed transcript analysis is recommended. Should redundancy in student responses be found, one must consider 
whether the prompts are the major influence or facilitation style and revise accordingly. Additional resources 
depicting postures, movement, and anatomy should be considered as well because students requested additional test 
preparation and video resources. Market research should also be conducted to determine the impact of the RYT 
designation on program enrollment. 

Assessment Tools 

Questions about test question and instructional resource alignment arose several times during the program’s 
pilot and evaluation periods. In order to validate the instructional alignment and assessment questions, a record of 
response attempts students made and results would need to be obtained. The assessments could either be tested with 
a student cohort or a limit on the possible attempts would need to be implemented along with observation of ensuing 
results. One instructor noted that she was disappointed with the quality of student posts. Further discussion 
transcript analysis should take place to determine if additional facilitator professional development is necessary or if 
the course structure needs revision. An anonymous feedback instrument embedded into each module might also 
help with identifying specific test question problems, organizational concerns, and better program tracking in 
general. 

Internet Learning Environment 

The request for additional instructions on the use of the tools embedded within the Moodle environment 
came up several times during the evaluation process. The evaluation process also revealed considerable student 
desire for the availability of additional communication features such as instant text and video chats. Since one of the 
main focuses of this program is to provide the foundations for building an active, informed, Christian yoga 
community across the country, the addition of these features is highly recommended. However, along with 
providing additional communication features, more robust training instructions should be provided for the use of 
these tools. The evaluation also revealed discrepancies with the Moodle learning environment organization; some 
students remarked at the excellent organization while others noted their desire for a revised organizational structure. 
Thus, more information should be gathered prior to any major organizational changes. 

Discussion 

Hakkin notes that instructional designers need to pay more serious attention in recognizing the importance 
of participatory and collaborative modes of designing needed to support explicit thinking about the border between 
the design artifact and its context of use (2002). This is not a new challenge to the field. Hannafin also wrote that 
instructional designers have not been very open to new alternatives; they have not reassessed the basic foundations 
or assumptions of the models (1993). The innovative instructional design project presented by these authors will 
address these common concerns and provide insight to new design and development strategies, which closely 
integrate the instructional program development with its intended context of use. 

It has also been noted that the interplay of different learning theories often provides a necessary challenge 
in the design of complex, collaborative learning environments (Hanafin, 1993). We feel the success of this project 
must be attributed in some part to a close connection between constructivist learning theories, traditional systematic 
design models, and the weaving of human performance technology interventions, which have resulted in a 
comprehensive, innovative learning program. For example, the authors agree that team buy-in is essential to 
achieving timely progress. Extra precaution and support must be provided to prevent client from asking coach to do 
all the work. Use subject matter experts in your coaching team to handle instructional design, technology 
implementation, and project management. Client feels ownership of completed instruction so client rejection is 
minimized. Impacted the organization’s outlook and approach to instruction throughout the organization. 
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Abstract 

With the popularity of smartphones on the rise, students are using their mobile devices to access everything 
from email to their social networks. The purpose of this research was to explore what student perceptions were on 
using their smartphones for their online courses and to explore what were distance course designer perceptions on 
developing courses for the smartphone device environment. Students were concerned about the costs involved, the 
limitation of the input device, the small screen size, lack of operational programs, the location of learning, and 
whether the device was the best tool to use. The students also felt that smartphones were constantly being approved 
upon and they had no doubt that future device improvements will overcome the current issues. The designers 
reported that there was compatibility issues between the various operating systems, that developing easy navigation 
was important, and that course content needed to be lightweight and portable. The evolution of the smartphone has 
undeniably affected the way we communicate, socialize, work, and access information. Through the perspectives of 
online learners and online course developers, this research explored the issues of it being a mobile learning device 
for online course access. 



Introduction 

The advancement of computer technologies and the World Wide Web have allowed many users to 
successfully fulfill degrees via the online learning environment. University systems have recognized that expanding 
their programs requires their ability to offer an online component (Allen & Seaman, 2006). 

A successful online program starts with basic essentials such as a functional and reliable campus network, 
secure access, data storage, and availability of the technology tools to access the online environment. Other factors 
that affect success are faculty training (for administrative and technology support), technology mentoring, and 
effective course development. Besides being the facilitator of an online course, the instructor must also provide 
students with the administrative and technology support (Maiuka, Shi, & Bonk, 2005; Restauri, 2004). Removing 
institutional barriers, updating obsolete policies, accepting the technology paradigm shift, developing faculty and 
staff skills, considering faculty workload, designing effective instruction, and selecting appropriate technology 
equates to a good online program (McLean, 2005; Thompson, 2003). The use of mobile technologies are becoming 
acceptable means from which students connect to their online courses. Using mobile technologies to learn lends 
itself to the term mobile learning or m-learning (Wagner, 2005). These mobile technologies include laptops, 
handheld computers, netbooks, and more recently devices such as tablet computers and smartphones. 

Wagner (2005) defines a smartphone as any handheld device that integrates personal information 
management with mobile phone capabilities, and the key feature of its addition of mobile applications. Mobile 
phones and emerging technologies are enabling phones to do more than transmit voice calls. Besides the basic 
ability to transmit voice communication, mobile phones are evolving in tandem with other technologies such as 
GPS, digital photography, digital video recordings, electronic book readers, and internet browsers. Whether for 
business or personal use, smartphones’ communication prowess includes the ability to surf the internet, instant 
message, email, and access social media tools. 

The International Telecommunication Union (2010) predicts that by the end of 2010, there will be five 
billion mobile phone subscribers worldwide. Pew Internet reports that 82% of all U.S. adults are cell phone users, 
with a little less than a quarter of them owning a landline (Purcell, Entner, & Henderson, 2010). Amongst all U.S. 
cell phone users, the Nielson Report (2010) has discovered that 25% of all U.S. mobile subscribers own a 
smartphone. More than half of U.S. teens are texting daily, and over 40 million U.S. adults are accessing the 
internet using their cell phones. 
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Advances in smartphone technologies have made it increasingly easy for users to stay connected. Within 
the last decade, these smartphones have become smaller, less expensive, and more powerful in computing. These 
smartphones have revolutionized the way individuals are connected to the world (Schlosser, 2002). Individuals are 
using the available features like the calendar, GPS, the web browser, the camera, the organizer, games, and 
thousands of other applications; it would seem logical that students with smartphones would use it to access their 
online courses. 

With the rich success of higher education implementing online programs, emerging research are now 
focusing on the advancement of technologies for use in education. The advent of Wi-Fi, Bluetooth, and global 
wireless technologies such as GPS, GSM, 3G, and 4G have created new possibilities for students to stay connected 
online. When the new communication technologies merged with mobile computers, mobile learning emerged. In 
terms of technology and learning, mobile learning has also been defined as learning that takes place via wireless 
devices such as laptop computers, personal digital assistants, netbooks, e-readers and mobile phones (O’Malley, 
Vavoula, Glew, Taylor, Sharpies, & Lefrere, 2003; Wagner, 2005). With the expansion of data networks and the 
advancement of smartphone devices, Asians and Europeans are finding that their computing and broadband 
computing needs can be met through a single device (Wagner, 2005). Given the vast research concerning m- 
learning devices, this research will specifically focus on the smartphone. 

Methodology 



Purpose of the research 

The use of smartphones to support, enhance, and improve online course access is a relatively new idea; 
many students are using their smartphones for their everyday communication needs. The purpose of this exploratory 
study was aimed to explore if students are using their smartphones for their online learning and what their 
perceptions were about using them and what online course designers felt about developing for them. 

Exploratory study 

This exploratory study on using smartphones for accessing online courses, did not utilize quantitative 
research techniques because it was not the best method for reporting perceptions. Literature recommends the use of 
a qualitative approach when confronted with new and emerging research (Maykut & Moorhouse, 1994; Hoepfl, 
1997). This study utilized focus group interviews and follow up one-on-one interviews. 

A typical description of this qualitative study suggested that it is based on grounded theory. Qualitative 
researchers using grounded theory are expected to gather rich descriptive data and ground conclusions and 
understandings from the data collected, not prior theories (Charmaz, 2003; Creswell, 2005; Straus, 1987). It is the 
particulars that will alert the researcher the attitudes of the participants towards the research topics. This qualitative 
inquiry involves using a flexible structure (Creswell, 2005). Basch (1987) has suggested that focus groups with 
adults are a relatively easy and flexible way to gather a diverse range of information and learn about the ideas and 
opinions of homogeneous groups. 

Instrumentation 

Several instruments were used to investigate the research topics. In allowing for emergence, the most 
effective strategy employed in this qualitative study was the use of triangulation to ensure reliability: focus group 
interviews and follow up interviews. 

The researcher first compiled and determined significant themes from each group about their perceptions of 
smartphones use or development for mobile learning. After the preliminary themes were developed, the researcher 
interviewed new participants for one-on-one interviews. 

The focus group phase of data collection functioned as the primary source of findings. The second phase of 
one-on-one interviews refined the data points collected in the first phase. This two-step data collection process 
supports the principles of constructivist grounded theory (Charmaz, 2003). This constructivist approach 
acknowledges that the researcher interacted with the participants, the data, and the analysis. The data collected was 
used to formulate common themes and ideas which were then reported in the findings section. 
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Participants 



The first topic of the study was to understand qualitatively student perceptions of using smartphones to 
access online courses. The second topic was to explore course developers’ perceptions and ideas behind developing 
online courses for the smartphone environment. The first phase of the study collected data from two separate focus 
groups. Each small group had at least three participants each and all participants had a smartphone device as their 
main cellular device. The focus groups were separated into students and course developers. 

The student group comprised of five volunteers who fit three criteria: having a smartphone, having 
participated in an online course, and be willing to participate in this exploratory study. Three males and two females 
were selected. All of the students were at least 20 years of age and attend a public university. Participants were 
classified undergraduate and graduate students in differing departments which included computer science, language 
studies, and education. The researcher purposely selected participants because of the polar criteria needed to 
explore the topic. 

The second focus group was comprised of four course developers from a grant sponsored online course 
development team. Three were instructional designers and the other was a graphic designer. The participants also 
came from various educational backgrounds from graphic design, communication, business administration and 
educational technology. 

The one-on-one participants consisted of two smartphone using students and two online course developers, 
and their ages ranged between 25 to 35. Their educational backgrounds also varied. The students reported that they 
each have had more than two years of online course experience. While the developers had more than five years of 
experience in developing online courses. The interviews were used to clarify some of the emerging themes and 
ideas collected from the first phase. 

Research topics 

The researcher began the focus groups with a grand tour question. Grand tour questions are broad and 
unfocused questions which allow the interviewees to lead the researcher into specific topics (Shank, 2006). The 
qualitative inquiry of the focus groups and one-on-one interviews aimed to gather information and explore two main 
topics: students’ perceptions of using smartphones for their online courses and course designers’ perceptions when 
developing online courses for the smartphone environment. Figure 1 lists sample questions used in the focus group 
and inteiviews. 



Topic 1 


What are students’ perceptions of using smartphones for their online courses. 


Topic 1 Sample Questions 


Are you accessing your online courses via your smartphone? 

Is using a smartphone to access your course a viable means to access your 
online course or course content? 

What do you perceive to be the important basic level needs (student needs, 
software needs, or hardware needs) required for accessing online courses on 
your smartphone? 


Topic 2 


What are course designers’ perceptions about developing online courses or 
course content for the smartphone. 


Topic 2 Sample Questions 


What should course developers know about developing courses for the 
smartphone environment? 

What are some of the advantages/challenges of developing courses for the 
smartphone environment? 

Should course designers be developing courses for use on a smartphone? 



Figure 1. Semi- structured topic and sample questions 
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Topic 1 Findings 

Using a smartphone costs students money. 



Findings 



Although the participants had smartphones, there are a couple of costs involved in using it. The first cost 
would be the purchase of the smartphone, and the second cost would be the monthly plan to use it. Some students 
perceived that if they were using their phone for educational use, they wanted to know if they could get cheaper 
educational rates for smartphone purchase or cheaper data rates. The students felt the smartphone costs were not 
equivalent to textbook costs, since smartphone data plans required 2-year contracts which was much longer than the 
duration of any course. 

Smartphones need better input devices. 

There were differing skill levels when it came to typing text on their smartphones. Some smartphones 
utilize touch text, others include a physical keyboard, and others use Swype. Swype is a text tracing application that 
allows for faster and easier typing input. A participant reported that having bigger fingers meant typing was a pain 
and if he had the choice, he would just wait to use a real keyboard on his laptop. It was clear that the participants 
were learning to become more effective smartphone typists, but they agreed that typing speeds were significantly 
slower compared to real keyboard typing. On the other hand, most students felt that they preferred using a touch 
screen over a ball mouse or touchpad. 

Smartphones need bigger screen sizes. 

Participants agreed that screen size was one of their biggest concerns about accessing their courses online. 
The students reported that their smartphones could easily log into their courses, but the courses were difficult to see 
and required extra scrolling and zooming just to navigate through the course. Screen size differed amongst the 
smartphones but the general perception was that the screens were too small to productively view the course. 

Not everything works on the different phones. 

Along with difficulty in navigating through a course, participants could not access certain content in their 
course. Differing smartphones have different capabilities. For example, some phones are able to watch Flash 
animations, while others cannot. Another example is document handling; some phones could easily download and 
view documents while others could not. A participant stated that most files were made to be used on the computing 
environment and the smartphone is not quite there yet. 

Using a smartphone for accessing and completing their online course was not the best mobile learning device tool. 

The participants perceived that a laptop with internet access was the mobile learning device of choice when 
it came to online course access and participation. They suggested that using a smartphone as a supplemental tool 
may be a better approach when it comes to mobile learning. They perceived that accessing components such as 
email, electronic readings, at a glance course calendar or schedule, and list of homework assignments would work 
better with the smartphone environment. This method suited most students’ needs because it would not be wasting 
battery life. They felt that mobile learning would be optimally reached if they used their Wi-Fi enabled laptops at 
their convenience. 

Students felt that not everywhere was a good place for learning. 

When suggested that commuting would be a good time to learn, many responded that they used that time to 
play games, update their social networks, listen to music, and read up on the news. Many felt that it was just easier 
to complete their online learning in places more suitable for learning (e.g. home, library, cafe, or quiet corner). They 
perceived that their smartphones keeps them connected, but there were limits to how much they need to be 
connected. Participants felt that they were willing to wait and access their courses when they were ready to focus on 
it. 

There are problems but it’s getting better. 
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Most agreed that smartphones could make their lives easier by supporting, enhancing, and improving their 
access to their social, work, and educational lives. They felt that the hardware and software were improving with 
every new version of their smartphone. The students agreed that once they have owned a smartphone they would 
never go back to using a regular cell phone. With the smartphone issues being resolved, the participants were 
enthusiastic about one day accessing their online courses via their smartphones. 

Topic 2 Findings 

Compatibility is a huge issue. 

The designers all stated that developing for smartphones is difficult because there are seven operating 
systems (Android OS, Apple iPhone OS, Symbian OS, Linux, RIM BlackBerry OS, Palm OS, and Microsoft 
Windows Mobile). Thus, the developers felt that building applications for courses would cost too much since there 
are no standards across the operating platforms. The developers perceived that smartphone application development 
would emerge new functionality and usability problems. 

The course developers stated that smartphones all had web access. Logically, they suggested that using 
web standards was the most realistic way of addressing compatibility issues. HTML5, CSS3, and Javascript were 
brought up by more than one occasion because they are web based computer languages and require only a web 
browser with internet access to operate them. Using HTML5, CSS3, and Javascript would also be smaller in file 
size because the interactive and dynamic components of a course could be embedded in the code. On the other 
hand, HTML5 is not standardized itself, but it will be in the near future. Currently, most browsers are being 
developed to handle HTML5. The general perception is to develop courses using web standards that are future 
proof, easy to develop with, dynamic, interactive, and deployable to all web browsers. 

Ease of navigation. 

With smartphones having limited screen size, the developers felt that easy navigation will be an important 
design principle to follow. The development of an easy to use navigation system will promote a better course 
viewing experience without the hassles of constantly searching for course materials. Course content organization 
will also help students efficiently and effectively navigate the course. 

Course content needs to be lightweight. 

Along with the smartphone portability, the course content also needs to be portable. The designers felt that 
containing instruction that is easy to get to, easy to use, effective in teaching, and accessible at any moment, will 
positively impact students’ experience and add to the value of using a smartphone for m-learning. This idea also 
means that smartphones should not heavily use processing power or resources to view the course content. 

There is a future in mobile technologies. 

The designers all felt that there will be a need for developing for mobile devices. A designer happily joked 
that mobile technology development meant job security. That implication is very true, because designing for m- 
learning requires new expertise or personnel to effectively develop, administer, and support this new learning 
environment. In summary, they felt that smartphones will eventually become widely adopted m-learning devices, 
and that most good designers will have already prepared themselves for it. 

Discussion 

The researcher thought that the emerging themes were relevant and applicable to anyone thinking about 
developing courses for the smartphone environment. Hopefully, the findings may be valuable to students or 
instructors thinking about mobile learning in terms of the smartphone. Attempting to deliver courses on the 
smartphone is a fairly new concept and the feelings towards its use for online learning are mixed. On one hand, the 
flexibility for students to instantly have access to their online course deliverables may be a desirable draw to mobile 
online learning. On the other hand, the technology is so new that factors such as cost and non-standardization of 
devices poses real threats to its adoption. Course developers need to develop courses that are standards based and 
m-learning friendly. 
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This exploratory study began with the uncertainty that smartphones were the newest mobile device tool 
used to access online course content. It was quickly realized that the issues behind using smartphones for m- 
learning were big enough reasons to not to use them. It is also important to be aware that building courses for the 
smartphone will require extra time in terms of development, deployment, and support. The researcher recommends 
that online course development for smartphones be developed according to functional web standards and to be 
aware that hardware and software issues currently hinder full deployment. 

In conclusion, smartphones are being used for more than just voice calls and chatting, they are essentially 
becoming pocket-sized computers. To further explore this phenomenon, future studies may include measuring the 
adoption rate of students using smartphones for their online courses. Other studies could also expand on this 
exploratory study and define more thoroughly the design principles associated with smartphone course development. 
The potential use of these devices in our personal, work and educational lives are being extended with users wanting 
and expecting more. Therefore, a mixture of hardware changes and computing standards will no doubt play a key 
role in one day offering fully online courses developed for the common smartphone. 
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Microblogging with University Students 24/7: Twitter Comes of Age 



Ingrid Graves 
Yadi Ziaeehezarjeribi 

Introduction 

This study presents the quantitative and qualitative results of a Universal Design (Behling & Hart, 2009) 
modification to a face-to-face blended learning environment which intended to quickly disseminate classroom 
information and provide for a diversity of learning styles. One of the most challenging components of teaching in 
higher education is on-going formative feedback between university professors and their students. According to a 
recent review of online learning studies by the U.S. Department of Education (2010), 

Online learning can be enhanced by giving learners control of their interactions with 
media and prompting learner reflection. Studies indicate that manipulations that trigger 
learner activity or learner reflection and self-monitoring of understanding are effective 
when students pursue online learning as individuals. Studies indicate that manipulations 
that trigger learner activity or learner reflection and self-monitoring of understanding are 
effective when students pursue online learning as individuals, (p. xvi) 

Additionally, Cleveland, et.al. (2002) notes that the blended nature of courses cannot leave the integration of 
technology to the technology courses. Adding blended learning components to student’s activities for both Online 
and Face-to-Face learning environments adding flexibility to fulfill course objectives (Graves & Ziaeehezarjeribi, 
2009). 



Literature Review and Background of Study 

After teaching undergraduate courses for a number of years, several professors became dismayed with an 
increase in the number of cell phone interruptions to classroom activities. While the university professors were not 
necessarily distracted by the ring tones, some of the students were. Seldom does a day pass when a professor did not 
complain to another colleague of the “rude” behavior of cell phone users in the classroom. Regardless of how 
instructors enforced classroom etiquette on the use of mobile technology (cell phones) during school year, the usage 
of cell phones during instructional time has increased. Out of consideration for those students with attention 
differences, many professors from where this study originated, began to include a standard statement in their 
syllabus regarding the use of cell phones (Figure 1). 

Figure 1. Standard Classroom Etiquette and Netiquette Statement included in Syllabus 



Professional courtesy also includes the creation of an environment by all learners and 
the facilitator that is conducive to learning. Please turn off cell phones, beepers, and 
watch alarms before class. 

If you are using humor or sarcasm, clearly label the message as such. i.e. “ha ha” or :) 

• Know your audience. Make sure that the person(s) to whom you are sending your 
message are the appropriate one(s) with whom to communicate. 

• Avoid cluttering your message with excessive emphases (such as stars, arrows and the 
like). They may make the message hard to follow. 

• If you are responding to a message, either include the relevant part of the original 
message in your response, or make sure you refer to the contents of the original 
message. 

• Be specific, especially when asking questions. 

• Include your name in the text of your message, as well as other contact information, 
such as email address or phone number for a reply. The end of the message is a good 
place for this information. 
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Digital Natives. The increasing frustration of university professors can be directly attributed to the shift in 
the way students currently experience and access learning. According to Skytland (2009), there are several 
important components (Table 1) indicative of contemporary learning styles. 



Table 1. Unique Qualities of Today's University Students 


On Demand 


Accustomed to wireless mobile technology, which 
provides instant access to others, for learning, social 
networking, which may blur the lines between school 
and other social settings. 


High Expectations for 
Learning Environments 


Expect high quality interactive media, able to multitask, 
learn through sound bites or short pieces of 
information. 


Global Connections 


Have access to instant feedback for questions through 
the internet, social networks. 


Collective Participation 


Many students learn through collaboration. 
Alternative communication may support increased 
engagement, (e.g. shy students may choose to submit 
answers electronically instead of verbally) 



Contemporary (Net-Generation) students are accustomed to instant access to high quality global 
perspectives and information, and collective participation in the information age via social networking. In short, 
students are accustomed to having a voice in their own learning. The increase and dependence on the use of Mobile 
Technology (cell phones) may have less to do with high quality access and more to cell applications that serve the 
same functions a laptop computer. For instance, many of the students, with whom this study was conducted, used 
their cell phones to “ Google ” answers to questions posed in the classroom instead of depending on their own 
memories after completing the assigned reading for the course. Many students routinely accessed the university 
management system (BlackBoard) through their cell phones to complete reading assignments, post to discussion 
boards, read communication with the course instructor, and check grades. These shared experiences at this 
university level reflected a larger trend, which has shown 69% of college students said they were more likely to use 
the phone than the Internet to communicate socially (Jones, 2009). In that same report, 61% of 18-24 year olds 
owned a cell phone, and 19% of American Internet users (median age 3 1 years) used Twitter in August 2009. In 
short, mobile technology is smart and has the same functionality as a laptop. 

From static to ecstatic. This paradigm shift is more than simply easy access to information. According to 
Bonk (2009), Twitter can be used as a form of “quasi-virtual apprenticeship,” which has shown to shift the dynamics 
of the classroom in very powerful ways. As Bonk (2009) noted, Twitter allows students to “shadow” professionals 
through “the movements of a well-known researcher in the field who uses Twitter for online interdisciplinary 
discussion. Such an activity gives them (the student) a close up view into the life of the expert” (p. 301). 

More importantly, Bonk (2009) noted the potential to “give small teams a sense of identity” in much of the 
same way study groups functioned in the past. Noting that cell phones are now an integral component of students 
learning, noting the high expectations students have for interactive media, and the motivation to increase 
engagement using collaborative learning in classroom discussions, the researchers in this study decided to 
incorporate the use of microblogging (Twitter) as an optional component of classroom participation. 

Methodology 

Background 

As part of a Universal Design (Behling & Hart, 2009) project through the university, a group of faculty 
began to meet to create and improve on their own practice through close examination of accessibility for special 
needs students in the classroom. After several months of examination of each other’s courses regarding such issues 
as Web Content Accessibility Guidelinesl.O (World Wide Web Consortium, 2010), each person decided to make 
changes with the intent of creating course content that was “accessible to the widest possible variety of students” by 
learning to use software which automated course material (Behling & Hart, 2009). One of those professors decided 
to incorporate Twitter into the course design to help support student engagement in the undergraduate Reading 
Methods course. In addition to using Twitter to access course information, the faculty member with whom this study 
was conducted, employed the use of Just in Time Teaching (JiTT) strategies (Novak, Patterson, Gavrin, & Christian, 
1999; Simkins & Maier, 2004) which posed Twitter questions prior to class to both assess student understanding of 
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course material and prepare students for classroom discussions. Twitter posts were also used to remediate any 
misunderstanding of course content or instructions through collaborative responses. 

Setting 

Data was collected in a university undergraduate reading methods course in three different campuses in a 
small southwestern teacher education program. An online (Blackboard) management system was used to submit 
writing assignments, send and receive emails, access reading assignments and media, post to discussion boards, and 
receive grades. Twitter was initially offered as an expedient method through which students could have ready access 
to the course instructor, correspond and collaborate with other classmates, and receive course updates. 

Participants 

The participants included the course instructor and 130 (129 female: 1 male) senior level undergraduates in a 
teacher-licensing program over the course of two semesters. 27 students were over 30 years of age. 102 of the 
participants were between 20- 24 years of age. While the classes were predominantly of European-American origin, 
eight students were of Hispanic-American origin (not ESL). 

Data Analysis 

Instructor and student participation ( Twitter posts) were quantified into two discrete categories, Classroom 
Discussions and General Information but were not limited to social discussions among peers. Pre and Post-testing 
data was collected to determine if Tweets helped student transfer important classroom content to various classroom 
assessments. For instance, the Writing Intensive component of the course held the potential to include information 
attained through following Twitter posts from professional educators. For instance, this study anticipated students 
would reference Twitter posts during writing assignments. An End of Course Survey ; was administered after final 
course grades were posted. 



Initial Findings 

Students were more likely to follow guidelines and instructions in short bites (140 characters or less) or 
cues (Morrison, Ross, & Kemp, 2004). For instance, the study protocol began the academic year with a short Tweet 
to all of the 130 students in the four course sections, “Do you believe there is no such thing as a lazy student?” This 
one-line opener laid the foundation for the manner in which students were expected to engage in classroom 
discussions. According to the professor, the tweets allowed students to consider the content related question while 
they completed the assigned reading for the week. Through tweets, students were more likely to focus on important 
information found on the end of course assessment. Instructors used feedback from before-class Tweets to 
individualize instruction for a particular section or make decisions on field placements (Table 1). 



Table 1. Faculty Comments/Instructions 



Classroom 
Discussion Questions 


“What is the difference between CALP and BICS?” 

“Be prepared to answer the question. How has writing 
instruction changed in the past 30 years?” 


Changes in Class 
Schedule 


“If you think the drive to (university town) is too 
dangerous for you do not attempt the drive. 


Course Reminders 


“Reading and Writing interview and one page 
discussion due on BBoard” 


Answers to Student 
Questions 


“Remember to bring your QRI-4 reading miscue 
analysis book and extra paper and pencils for students. 
See you at noon at (school name)” 

“All of your assignment due dates are listed in your 
syllabus.” 



The data from this study mirrors the benefits of JiTT (Simkins & Maier, 2004), “students come to class better 
prepared, leading to more effective in-class discussions and activities” (p. 3). The ability to communicate with the 
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professor and other classmates created opportunities for instant updates on classroom information and remedy 
misunderstandings in classroom assignments or due dates. 

Twitter allowed for meaningful ‘‘teachable moments” throughout the semester. 



Table 2. Student Participation 


Answers to 
Classroom 
Discussion 
Questions 


“Vowel followed by r makes special 
sound. ..star.. ..bird” 

“Gee talks about how video games are created with 
methods to improve thinking and complex problem 
solving” 


Questions about 

Classroom 

Assignments 


“WOW! ! ! What a week. Let's hope my (assignment) 
will get a good grade this time. I feel better about it 
this time around thanks to (professor's name). 


Questions about 
Field Placement 


“Does (section name) meet at the elementary today?” 


Student to Student 
Posts 


“There's a really bad accident on (highway) before 
(town). Both lanes are blocked. I'm stuck. .be careful 
everyone!!!!” 


Other 


“This text is from the (school district). Due to current 
weather conditions all school campuses will be closed 
today, Thursday F ebm ...” 



Pre and Post-Testing 

Quantitative (Pre and Post testing) data revealed a significant increase in the use of discrete course content 
vocabulary, such as basic interpersonal communicative skills (BICS) and cognitive academic language proficiency 
(CALP) (Cummins, 1981). Pre and Post data showed that students had an 87% mastery of questions, which were 
Tweeted compared to a 43% mastery of those which were not. Because there was no experimental design to control 
for contributing factors, such as the difficulty of different course content objectives or different methods of 
instruction between sections, formative claims surrounding pre and post testing cannot be made. In future studies, 
the need to find direct correlations to improved retention and control for external variables are in order. Carefully 
placed tweets may have allowed students to understand content in preparation for the teacher-licensing exam. On the 
End of Course Survey, four students noted that without Twitter they would not have remembered important 
information for a quiz. A less successful endeavor was the suggestion for students to follow blogs by noted 
professors such as Stephen Krashen and Kenneth Goodman. Only two participants incorporated comments from 
blogs into writing assignments or classroom discussions. 

End of Course Survey 

Qualitative post semester interview data revealed that 82% if the participant self-reported they felt more 
accessible to the university professor through instant feedback for questions regarding classroom projects, difficulty 
with posting assignments, instant notification of changes to classroom schedule (weather), and fieldwork. Students 
Twittered with each other about everything from classroom projects to social networking, including family issues, 
professionalism, and yoga meeting times (Figure 2). 
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Figure 3. Student Tweet 




For today's homework, do we have a 
section of our ESL paper due today? 
Or are we just bringing our articles to 
class? 






Unfortunately, 3% of the participants reported Twitter was an invasion of their free time, felt they had 
“already done their time in (name of professor’s) class,” and “resented having to worry about participation points” 
outside of class time even though the use of Twitter was optional. These comments came from the same participants 
who self reported engaging in less than 1-3 hours of out of class work during the week. Because the course 
instructor did not require Twitter, post course interviews revealed students who opted not to participate on Twitter, 
felt left out of classroom communications (Figure 4) even though similar content was posted on the mandatory 
university management system 

Figure 4. Instructor Feedback 



ft 



found on NBC 
Education 



Excellent! Go to Education Nation Senes 
The network is doing a week long focus on 



n 



ur Country Our Future has to be one of 
the saddest videos I have seen Common core standards’ I would 
like to know more 



Other F indings 

Twitter had fewer interface problems than did the university management system. For instance, when a 
severe weather alert was posted, most students were not near computers but had access to cell phones. Additionally, 
when students did not have cell phone reception, they were still able to access the professor through text messaging 
capabilities of Twitter. The professor noted, “ft is so much easier to send one bit of information to a hundred 
students through my cell phone, than the slow process of driving back to my office, turning on the computer, 
opening up my (university management system), accessing each section individually, and then composing four 
different emails. That alone makes Twitter important as classroom management tool.” 

While Twitter did have fewer interface problems, students encountered several small problems after 
signing up for an account. Future integration of Twitter into the normal class routines will include specific time 
devoted to testing student accounts and in-class opportunities to troubleshoot software settings. While the 
instructional video (Howcast, 2010) used with this course was required, at least 13% of students struggled with the 
Twitter settings because they had not watched the video. Moreover, several students encountered problems with 
their cell phone provider and did not receive text messages until hours or days later (Figure 5). 
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Figure 5. Student Difficulties with the Use of Twitter 



also wanted to let you know. Twitter finally got my 
I account cleared for text messages Apparently it was blocked due 
to excessive tries 



The limited number of characters (140) compelled students to synthesize important information during 
classroom discussions, brevity did not always allow students to complete a thought before another person added a 
comment. Educators must remember Twitter was not used to take the place of regular in-class written assignments. 
On the contrary, the need for brevity included extensive Internet slang (e.g. lol, U 2, :), etc). Some instructors may 
find the slang difficult to decode or not appropriate for the university setting. When considering implementation into 
the classroom, clear Internet etiquette should be included in the course syllabus. Several students made inappropriate 
and unprofessional comments to the course instructor during their twitters, which affected their professionalism 
grades. The informality of Twitter may have blurred the lines between formal and informal vocabulary. 

Conclusion 

For the instructor of this study, Twitter served as a tool to integrate all the important qualities of Universal 
Design (Behling & Hart, 2009) and Just-In-Time Teaching (Novak, Patterson, Gavrin, & Christian, 1999; Simkins 
& Maier, 2004) taking into account the unique qualities of today’s university students. Adding new tools to student’s 
daily activities provide students with the flexibility of time and place which is not a substitute for replacing a teacher 
or a quality lecture, however, provide the students the potential for transferring information synchronously and 
asynchronously. This is mainly to apply tools that are widely used by students in their daily activities through social 
networking and mobile technology at their disposal. Future integration of Twitter into the classroom will still need to 
take into account students who do not have access to Web 2.0 applications and instructors who are not comfortable 
with 24/7 access to students. The definition of Blended learning tools should include the important application of 
mobile communication devices. 
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Model of Pedagogical Framework and Software Architecture for Integrated 
E-Learning Environment in Chinese Universities 



Xibin Han, Yingqun Liu, Qian Zhou 

Development and Problems of Applied E-Learning Systems in Chinese Universities 

E-learning systems were first introduced into Chinese universities and colleges in the late 1990s to 
supplement traditional teaching. The development and application of e-learning systems are partly derived from the 
internal needs of the universities and colleges. At the very beginning, only the single-purpose learning management 
system (LMS) was used to assist lecture-centered instruction. As teaching approaches became diversified and 
information and communication technology became popular, various e-learning systems were established and 
applied. These included collaborative activity-based learning, laboratory-based instruction, resource-based learning, 
and degree thesis-based research (Han et al., 2008). In addition, the Chinese government has played a very big role 
in the development and application of other related systems. As one measure to improve the teaching quality of 
higher education, the Ministry of Education of China initiated the Teaching Quality and Reform Plan in 2003 
(Ministry of Education of China , 2003). This created such special projects as specialty structure adjustment and 
authentication, course evaluation, teaching resource development and sharing, practical-based teaching, and training 
mode innovation (Ministry of Education of China, 2003). Many achievement demonstration and sharing systems, 
such as course demonstration websites - whose content is actually part of the above e-learning systems - that have 
been developed and used by universities and colleges have been evaluated for selection into the National Quality 
Course Plan by the Ministry of Education (Han and Liu, 2010). 

However, the above-mentioned e-learning systems, specialty/course demonstration and evaluation systems, 
as well as resource sharing systems were built in isolation at each university. As a result, instructors had to 
repeatedly create the same teaching resources in different systems for different purposes; students had to log into 
different systems in order to access relevant learning resources and tools; curriculum administrators had to collect 
statistical data from different systems and combine the information manually to get an overall assessment of 
students’ academic performance and instructors’ teaching performance. Moreover, updated teaching resources and 
results accumulated during daily teaching and learning activities and stored in the learning systems, could not be 
holistically included in the demonstration and resource sharing systems for the public in time. Therefore, those 
individually developed single-purpose systems would become obsolete after a short time because of the limited 
integration (Han et ah, 2008). 

A number of researches discussed the integration of multiple learning systems. Dimitrova et al. (2004) 
stated that flexible pedagogical frameworks were needed to underpin e-learning environments in order to ensure that 
they effectively addressed the individual learning approaches of two types of learners: those relying extensively on 
social interaction, and those most in need of lecture -based learning. Students should be given a choice of learning 
activities and tasks that support the development of different cognitive skills and promote meaningful online 
communication (Dimitrova et ah, 2004). The model in the study can be used to construct a learning environment 
integrating both learning approaches. 

Marengo et ah (2007) presented an idea to integrate LMS, videoconferencing, multimedia classrooms, and 
multimedia and management software, as well as MS Office, antivirus, editing, and streaming software, etc. into one 
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project: Advanced Services for E-Learning in Puglia financed by the Italian Ministry of Universities (Marengo et al., 
2007). However, the study did not describe the pedagogical and technical model indicating how these systems were 
integrated. 

Abdallah et al. (2003) presented a model for a learning environment that built several pedagogical baselines 
of e-learning strategies and supported the exchange of learner profile and courseware with other systems and 
repositories (Abdallah et al., 2003). The study did not consider the data exchange of learning systems with other 
related systems, such as curriculum evaluation system, academic affair information management systems, etc. 

These researches discuss the integration of multiple learning systems only from the view of a single course, 
or learning activity even. While the utilization of e-leaming systems scale up from the curriculum to institutional 
level, the number of involved systems increases dramatically, thus further complicating the issues of integrating 
single-functioned and isolation-built e-leaming systems. 

This paper discusses a model of pedagogical framework and software architecture that integrate diverse 
isolated systems into a whole learning environment. The objectives of an integrated e-learning environment in a 
university include: 

• the flexibility for instructors to roam among different systems and organize instructional resources and 
activities effectively depending upon their pedagogical strategies. 

• the convenience for students to access learning resources and tools across multiple systems. 

• the ability for curriculum administrators to evaluate the overall teaching and learning performances and 
choose valuable instructional resources and achievements for demonstration and sharing. 

• the ability for institutions to make systematic planning of e-learning systems and run a long-term 
sustainable development of the e-learning environment. 

Pedagogical Framework of the Integrated E-Learning Environment 

According to the desired organizational mode of teaching/learning in a university as well as information 
and resource exchanging among various systems, the proposed model is divided into three layers from the inside to 
the outside, and thus, constitute an integrated e-learning environment (Figure 1). 

In general, there are two ways for college students to obtain the credits required for graduation: first and 
foremost is curricular learning, the other is non-curricular learning, such as degree thesis research, resource-based 
learning etc., which need to be supported by the e-learning environment. 

At the very core of the environment, systems are centered on the curriculum. Different systems are used at 
different stages for supporting different course approaches. When different systems are required in one single course, 
it is necessary for students and instructors to share teaching information and resources, such as student information, 
teaching team information, course syllabus, learning resources, performance assessment and course administration, 
etc. 

The middle-layer systems support the non-curricular learning activities, which include the degree 
thesis-based research, student research training (SRT), and student self-learning, such as blog community-based 
learning and resource-based learning. Because these learning activities are carried out on an individual level, these 
kinds of systems do not need to exchange basic course information but learning resources. 

The outermost systems do not correlate to daily teaching and learning activities, but are platforms for 
demonstrating, sharing and evaluating the achievements and resources from the inner-layer systems. The forms of 
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demonstration for teaching activities and resources are different due to different requirements. For instance, a quality 
course project requires course-based demonstration of the curriculum construction achievements that is reviewed by 
the evaluators (Han and Liu, 2010). A featured specialty project requires a specialty-based demonstration of related 
courses (Ministry of Education of China, 2003). Teaching/learning resources from all these learning systems need to 
be collected into and managed in a unified repository. As a result, it is necessary to develop different support 
systems. 




Figure 1 Pedagogical framework of the integrated e-learning environment 

Software Architecture and Key Problems for Construction of Integrated E-Learning Environment 

The software architecture necessary to construct an integrated e-learning environment involves four layers: 
basic data, shared instructional resource, application system and portal (Figure 2). 

The basic data layer is built according to specific data standards. It provides basic data such as student, 
instructor and course information, and the data exchange interfaces with the application systems. 

The shared resource layer manages resources via a management platform. It gives application systems a 
uniform interface, which offers users convenient access and exchange of teaching and learning resources. 

The application system layer consists of the e-learning systems mentioned above: curriculum-based 
systems, non-curriculum-based systems, and demonstration, sharing and evaluation systems of the achievements and 
resources. 

The portal layer integrates various application systems. According to the organizational mode of 
teaching/learning in a university, users in the e-learning environment are divided into students, teachers, 



131 



administrative personnel, evaluating experts and visitors. Through the portal, they log into the integrated e-learning 
environment with ID-based user authentication to access personalized information and services corresponding to 
their roles, which enables their single sign-on and multi-system 
roaming. 



User role 

User authentication 



Student 



Instructor 



Adminstrator 



Evaluator 



Visitor 



System layer 
Portal 



Application systems 



Instructional resource repositories 



Management infomation databases 



Task/application 

Personalized information and 
services for user 



Curriculum based systems 
Project based systems 
Systems for demonstration, 
sharing and evaluation of the 
achievements and resources 


Resource management 
Resource sharing 
Resource exchange interface 



Data management 
Data exchange interface 



Figure 2 Software architecture of the integrated e-learning environment 



The software architecture of the application systems shown in Figure 3 illustrates how to develop various 
application systems based on component technology and how to integrate them into a whole. At the bottom lies the 
system component layer, which provides unified user authentication and role-based permission control for the upper 
layers. The middle layer comprises a variety of common functional components that are managed by a management 
module and called up by application systems in the top layer. When a new function is needed in the application 
system, the corresponding components are plugged into the system. The component-based architecture ensures the 
e-learning environment is technically scalable, thus making it possible to be easily customized to meet the individual 
building needs of a university and its dynamic expansion needs at different development stages. The J2EE technical 
approach is adopted during the development of the e-learning environment. 

According to the pedagogical framework of the integrated e-learning environment, four technical issues 
need to be considered to integrate the application systems in the top layer shown in Figure 3. 

Integrating different teaching approaches in a single course: Both teaching practices and theories 
indicate that teaching activities require application of various pedagogical strategies, including a combination of 
face-to-face and technology-mediated instruction, known as a blended instructional approach (Graham & Dziuban, 
2005). It is necessary for the instructor to select the appropriate teaching approach at different stages or for different 
contents within a single course. In the e-learning environment, two typical instructional systems - the 
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lecturer-centered and the activity-based - are integrated to support instructional activities. Since the two systems 
share course data (such as student information, teaching team information, course syllabus, leaning resources, 
performance assessment and course administration etc.), they are usually integrated into one course-based system 
through shared data tables and JavaBeans (Wikipedia, 2010). 




Figure 3 Technical framework of application systems in the e-learning environment 

The example in Figure 4 shows two typical instructional systems that are integrated to support instructional 
activities in a course called Software Engineering. With the lecturer-centered learning system, the instructor 
publishes the syllabus, calendar, notice and learning materials to the students, communicates with them in the 
discussion area, and evaluates their learning performance with assignments and exams, etc. At another stage when 
tasks are assigned to different groups of students, the activity-based learning system is used to assist in this kind of 
problem-based learning, such as the design and development of a book information management system, which, in 
the example shown in Figure 4, is selected by six student groups. 

Integration of multiple teaching/learning activities centered on a single instructor or student: The 
instructors and students in universities may involve various teaching/learning activities. It is not viable for 
instructors and students, as users, to select and log onto the systems one by one and then make relevant operations. 
Instead, they select appropriate systems for different teaching/learning activities from a unified user interface. 
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Since the non-curriculum based systems share only a few course data with others, they are integrated with 
other systems through a loosely-coupled mode, i.e., existing as independent systems and exchanging information 
through data interfaces. The example in Figure 5 shows, after logging on, an instructor can enter his personalized 
teaching and learning environments to make selections across systems of course teaching, degree thesis based 
research, course demonstration, quality course application and instructional resource management and utilization. 
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Figure 4 Integration of two instructional approaches in one course 
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Figure 5 Integrated user interfaces of multiple systems in the e-learning environment 

Sharing and quoting instructional resources across different systems: Teaching and learning resources 
generated during the daily teaching process based on different systems are collected and managed in the resource 
management system, and shared among multiple systems through a uniform interface. The example in Figure 6 
shows the instructor editing learning materials within a course with the instructional system quoting resources either 
from the resource management and sharing system, or directly from the quality course application and review 
system. 
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Figure 6 Sharing and quoting learning resources among different systems in the e-learning environment 

Exchange of basic data with other related application systems: Basic data, such as information about 
students, teachers and courses, come from two sources: the unified data center in a digital campus, and related 
systems like the academic affairs administration system shown in Figure 7. 
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Figure 7 Exchange of basic data with other related application systems 

Integration of outermost systems and inner-layer systems: The achievements and resources for the 
specialty and curriculum are demonstrated and shared through the outermost systems, which are acquired from the 
inner-layer systems through the data and resource interfaces. The example in Figure 8 shows the course list in the 
bachelor degree program Design, Manufacture and Automation of Mechanized Systems being demonstrated through 
the specialty and course demonstrating system. The course list is acquired from the academic affairs administration 
system and the course resources from the instructional system. 
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Figure 8 Integration of outermost systems and inner-layer systems in the e-learning environment 

Application of Integrated E-Learning Environment in Chinese Universities 

A software system called THEOL developed on the basis of the above pedagogical framework and 
technical architecture of the integrated e-learning environment has been implemented in over 60 universities in 
China. THEOL includes the systems discussed above, six of which are listed in Table 1. The dates in the table 
indicate when the systems were installed in five universities with different rankings located in different geographic 
areas in China. 
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In actual fact, the e-learning system was installed in higher ranking universities before the lower ranking 
universities due to differences in funding, faculty quality, IT popularity, and so forth. However, no university can 
install all these systems in one go but rather over a period. The software architecture of THEOL is effective at 
integrating different systems installed on different dates into one whole system. 



Tablel Implementation of THEOL in some Chinese universities 



University 


Course based 

Instructional 

Systems 


Non-course based 

Instructional 

Systems 


Demonstration, 
Evaluation & Sharing 
Systems 


Location, 

Ranking** 


Lecture- 

centered 

course 

learning 

sys. 


Activity 

-based 

learning 

sys. 


Degree 

thesis 

based 

research 

sys. 


Resource 
manage 
ment & 
sharing 
sys. 


Quality 
course 
application 
and review 
platform 


Course 

and 

specialty 
demonstra 
ting sys. 


Nankai 


2002.12* 


2006.3 


- 


2002.12 


2006.3 


2006.3 


North 
China, 13 


Northwest 

A&F 


2006.4 


2007.3 


2010.3 


2006.4 


2009.8 


2009.8 


Northwest 
China, 54 


Nanchang 


2003.1 


2006.8 


- 


2003.1 


2006.8 


2008.7 


Central 
China, 71 


Y angzhou 


2007.2 


2007.2 


- 


2007.2 


2008.2 


2008.2 


East 

China, 102 


Chengdu 


2007.12 


2007.12 


- 


2008.10 


2007.12 


2008.10 


Southwest 
China, 262 



* Note: The date indicates when the system was installed on the university’s server. 

** Note: The data are quoted from China’s university ranking announced by the CUAA.NET and the 21 st 
Century Talent Weekly in January, 2010 (Zhao, et al. 2010). 



Conclusion 



While the utilization of e-learning systems can be scaled up from curriculum to institutional level, the 
issues of integrating single-function and isolation-built e-learning systems needs to be considered. This model of 
pedagogical framework and software architecture has been proposed to integrate disparate systems into a single 
whole learning environment to facilitate diversified e-learning strategies and information exchanges across the 
systems. The proposed model and principles will be helpful in designing and developing the integrated e-learning 
environment. 

Universities in China differ vastly in terms of the amount of funding, faculty quality, IT popularity, and so 
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forth. As a matter of fact, the number of e-learning systems constructed differs as well. However, it is obvious that 
the integrated e-learning environment would eradicate the problems of the isolated systems. 

The model supplies a pedagogical foundation which allows instructors to integrate ICT into their teaching 
by different systems, and also supplements students’ self-paced learning with a wide range of resources and tools. 

The component-based technology and scalable technical framework are conducive to being customized for 
different universities, and could withstand dynamic expansion needs at different development stages. 

The model contributes to the systematic planning of e-learning systems and the long-term sustainable 
development of the e-learning environment. 
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Using Technology for Reflective Practices in Teacher Education 
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The teacher education program at a large university has incorporated technology in its 
curriculum design by integrating technology into theory and practices courses and 
culminating with student teaching supervision. To use best practice in technology for the 
teacher education programs, faculty members collaboratively created opportunities for 
experiencing learning through an online/hybrid course and the utilization of video and 
audio recording of teaching performance followed by self-analysis and reflection. 



Today’s preservice teachers become professional educators who teach Kindergarten - 12 th grade students 
in schools across our nation. These teacher candidates need to be prepared for a rapidly changing world in which 
five out of the ten fastest growing job areas will be computer related (College Planning Network, 1996). The 
National Council for Accreditation of Teacher Education (NCATE) report titled “Technology and the New 
Profession Teacher: Preparing for the 21 st Century Classroom” illustrates the technology skills and knowledge that 
new professional teachers must acquire in the teacher education programs in order to be prepared for the 21 st century 
classroom. 

In a study designed to determine the most effective methods of technology integration (Sara & Eric, 2003), 
the authors found that infusion throughout a teacher education program, including field experiences/student 
teaching, rather than separate training was the most effective method. Recognizing the need to incorporate 
technology into teacher education programs, universities are beginning to address the training of their preservice 
teachers in the use of classroom technology in more progressive ways. Some teacher education programs have added 
a course specifically designed to educate teacher candidates in technology skills and integration strategies (Hargrave 
& Hsu, 2000). Others have selected to address technology by integrating its use into existing courses in the teacher 
education program (Cherup & Linklater, 2000). 

Rationale 

In order to better prepare future K-12 teachers on the use of technology for instructional purposes, the 
university teacher education faculty must continue their own professional development in this critical area. The 
teacher education program at a large university has begun to address technology by integrating its use into existing 
theory and practice courses, student teaching seminars, supervision of student teachers, and induction courses. Most 
instructors are using varied applications of technology integration in their classes and minimal, if any, collaboration 
and dialogue exists across faculty, even though they are teaching the same courses. To potentially standardize the 
goal of using best practice in technology for the teacher education programs, the faculty members who teach the 
student teaching seminars and capstone and induction courses have created an online/hybrid course for student 
teachers and first year teachers supported by the induction courses. Online/hybrid options for these courses are ideal 
for the student teachers and current educators who are located at long distances from campus and who have various 
school related commitments. This represents a new student population eager to learn via technology. Technology 
also allows for student teachers to apply reflective practices via the use of video and audio recordings. 

There are three major goals for creating capstone/ induction/student teaching seminar courses online. First, 
novice teachers will be able to acquire technology skills and knowledge to be prepared for the 21 st century 
classroom during the course work. Secondly, faculty members in the teacher education department will design and 
teach capstone/induction/student teaching seminar courses which align with National Educational Technology 
Standards (NETS) and NCATE standards in technology so that faculty members and the school administrators can 
evaluate their teacher candidates’ technology acquisition levels more effectively at the end of the program. Third, 
this course will be a model of a quality online course with the best practice of classroom technology which will 
impact other teacher education courses and programs. 
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Timeline of Activities 



Spring, 2009-teacher education faculty conducted needs assessment of technology with local partner school 
teachers (e.g. what kind of technology is most needed to enhance your teaching and student learning, etc). The 
faculty members piloted the technology curricular components in their seminar and induction courses based on 
technology standards and teachers’ technology needs and made appropriate changes. 

Fall, 2009-Winter, 2010- teacher education faculty taught student teaching and capstone/induction courses either 
online or hybrid method. The faculty ensured that the course requirements (creating student teaching videos, online 
discussions, and etc) reflect current technologies and teacher practices using technology. Student achievement was 
measured based upon the exiting school of education elementary standards modified by National and NCATE 
standards in technology. These standards verified that each of its student teachers is competent in creating and using 
instructional technology to enhance teaching and learning, including teacher candidates’ communication and 
collaboration, research and information fluency, critical thinking and problem solving, and Digital citizenship. 

Also, teacher education faculty will evaluate the standardized technology online/hybrid course from student teachers 
and the 1 st year students’ feedback. 

Spring, 2010-Faculty added video as an additional reflection tool for student teachers. 

Results/Conclusions 

1) What kind of online resources were created in the Hybrid and Online Student Teaching Seminar 
Courses? 

The data will be collected and analyzed as a result of the faculty participation in the activities listed above. 

2) How did capturing student teaching video help student teachers reflect on their lessons? 

Preliminary data is positive, demonstrating a positive impact on the reflection practices of the student teacher. One 
author reported on Student A’s pre-reflection (lesson: Tomas Edison Story) and post-reflection (after the video 
watching). Student A also wrote about the benefit of watching her video to reflect her teaching so we can use that 
one as a conclusion. The following is an example of qualitative comments emerging in the data: 

Student A said, 

I believe being able to watch myself on video allowed me to have a better concept of my teaching abilities. 

1 think the video was extremely helpful. You always have an idea or picture in your mind of how the lesson 
went but watching yourself allows you to relive it from a different perspective. 1 think it reconfirmed what I 
believe my strengths are and allowed me to see what more I have to work on. I am grateful for the 
opportunity to see myself on video and I think it is an extremely important and helpful to see yourself from 
the outside after the assignment. 

Student B said, 

At first, I was really unhappy about the way the lesson went. 1 thought the students were completely out of 
control and not paying attention. After watching the video I realized, the lesson was not that bad. The 
students were interacting with each other and communicating their ideas. If I were to do the activity again I 
would put the chart on the overhead. My back was to the class too much and maybe that’s where I lost 
their attention and had to settle down the class. I also noticed it took a significant amount of time to 
transition from overhead to the lesson itself. Again, I need to have very clear prompts to guide the students 
through transitions. I should not assume they know to put their folders away and look at me in the front. I 
have working on having more of a presence in front of the class to easier transitions. 

Student C said, 

I believe being able to watch myself on video allowed me to have a better concept of my teaching abilities. 

I think the video was extremely helpful. You always have an idea or picture in your mind of how the lesson 
went but watching yourself allows you to re live it from a different perspective. I think it reconfirmed what 
I believe my strengths are and allowed me to see what more I have to work on. I am grateful for the 
opportunity to see myself on video and I think it is an extremely important and helpful to see yourself from 
the outside after the assignment. 
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Student D said. 

It helped me to be aware of my body language. There were a lot of things going on in the classroom during 
that time that were distracting me, but I think I held my own fairly well. I wanted to students to be more 
self directed in this experiment but 1 can clearly see myself feeding them the information because I felt 
pressured on time. 

To use best practice in technology for the teacher education programs, these faculty members are creating 
and studying their collaborative experiences with online/hybrid course and utilization of video for reflective 
practices for teacher candidates. 
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Sharan Merriam (2009) remarks that “qualitative research has become a mature field of study with 
its own literature base, research journals, special interest groups, and regularly scheduled conferences” (p. 
vii). Many researchers who employ qualitative methods prefer to wade through piles of interview 
transcripts and collections of artifacts, but there are software tools available to help with the data analysis 
process. The American Evaluation Association ("Qualitative Analysis," 2010) lists thirty-five software 
tools designed for analyzing various forms of qualitative data: text, audio, and video. These tools range in 
price from free to thousands of dollars depending on a user’s licensing and feature needs. Some tools 
simply assist with the organization of data or the coordination of tasks between research team members, but 
some provide features that may aid researchers with data analysis. One of those features, data or 
information visualization, is the topic of the present discussion. 

The phrasing “data visualization” is used in the title of this article to stress the word data. The 
focus of this discussion is the possible usefulness of tools to visualize text data collected during research 
studies conducted with qualitative methods. There are related and overlapping fields of study involving 
visualization (e.g. scientific visualization, data visualization, and information visualization). The most 
appropriate of these fields for the purposes of qualitative research involving text artifacts is information 
visualization. Information visualization is defined as “the use of computer-supported, interactive visual 
representations of data to amplify cognition” (Card, Mackinlay, & Shneiderman, 1999, p. 6). The purpose 
of information visualization is to “communicate complex ideas to its audience and inspire its users for new 
connections” (Chen, 2010, p. 387). 

Card, Mackinlay, and Shneiderman consider the foundational period of information visualization 
to be the period from the introduction of the Silicon Graphics workstation in the 1980s to the end of the 
1990s when desktop personal computers’ standard configurations became capable of displaying 
sophisticated graphics, indeed, what many would consider seminal work (e.g. Tufte, 1983), in this area 
took place in that time period, though Tufte’s work involved static graphics. Visualization is still a topic of 
interest with David McCandless (TED, 2010) recently offering it to audiences outside of visualization 
researchers as a solution to information overload, which will allow us “to see the patterns and connections 
that matter.” 

The notion of using visualization techniques in qualitative research is not new. For example, 
Ruberg & Moore (1994) and Horney (1994) applied visualization techniques to analyze discussion data. 
Some recent examples of visualization in qualitative analysis can be found in the areas of expertise 
development in online games (Chen, 2009) and the teaching of writing (Sorapure, 2009). The adoption of 
visualization techniques in qualitative research though has not become widespread, and some qualitative 
researchers have recently noted a need for visualization tools (Slone, 2009). 

Authors have noted benefits of using visualization techniques in qualitative research. Horney 
notes that visualization techniques have highlighted “areas where data analysis should be reconceptualized” 
(p. 39) and can “lead researchers into deeper and alternative investigations of their data” (p. 39). Also, 
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visualization techniques for qualitative research data “may address transferability and confirmability” 
(Slone, 2009, p. 490). 

The practice of qualitative research may be a mature field as noted by Merriam (2009), but it 
continues to evolve. The incorporation of new technologies is one new and expanding area for qualitative 
research. Chen (2010) notes that research in information visualization using emerging, online, social 
networking tools is just beginning. The Merlien Institute is organizing an international conference in 2010 
("International conference on Qualitative Research in Web 2.0," 2010) aimed specifically at examining the 
intersection of Qualitative Research and emerging Web 2.0 tools. 

Writer Darcy Dinucci (1999) introduced the term web 2.0, which was later popularized by Tim 
O’Reilly (2005). Web 2.0 is a philosophy of software design describing “many individual tools that have 
been created with web collaboration, sharing, and/or new information creation in mind” Oliver (2007, p. 
55). Many articles have been written about the benefits, or potential benefits, of using Web 2.0 tools in the 
teaching and learning process, and instructional technology researchers have been key contributors to this 
area of the professional literature. Qualitative research methods are valuable tools frequently used in 
research and evaluation of educational and instructional technologies. Therefore, it is time to also apply 
these new tools to research and evaluation in the field of instructional technology. “Web 2.0 infovis 
[information visualization] applications can enable us ... to producing visual representations of 
information” (Sorapure, 2009, p. 60). The use of free, online tools allow users to “come to greater insight 
about the text” (Sorapure, 2009, p. 62). Examples of web 2.0 tools that show promise for use in qualitative 
research include: wordle (www.wordle.net), IBM’s Many Eyes (http://manyeyes.alphaworks.ibm.com/), 
TextArc (http://www.textarc.org/), and TAPoR (http://portal.tapor.ca/portal/portal), the Text Analysis 
Portal for Research. These tools can be as simple as word cloud creators (e.g. Wordle) which use simple 
word frequencies to more complex tools aimed at identifying themes in text (e.g. TAPoR). 

The utility of simple tools such as the word cloud generator Wordle for qualitative research may 
be questioned, but it has proven useful in at least one related context. Sorapure (2009) utilized word clouds 
in her writing courses and reported that her students were led to “reexamine the influence of her [the 
student’s] own bias” (p. 64) in an analysis of two text artifacts. This shows that there may be potential for 
qualitative researchers to use it for similar purposes while analyzing data. It is time to design studies to test 
the efficacy of using tools like the ones mentioned here for qualitative research. 

The advent of computer tools and capabilities for creating information visualization appear to have 
advanced ahead of theoretical groundings (Chen, 2010). As such, the field is an excellent fit for 
Instructional Technology researchers who are typically viewed as working in a linking science between 
fields such as Educational Psychology and practitioners. In this venue, Instructional Technologists are 
positioned well to apply Educational Psychology, emerging online tools, and qualitative research methods 
to achieve the “insight” (Chen, 2010, p. 388) information visualization hopes to offer its users. 
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Abstract 

This project explored mandatory features of an e-portfolio system for teachers to construct electronic portfolios 
effectively, especially for students with intellectual disabilities. Further, a special education e-portfolio system was 
developed according to the explored features by modifying the open source Moodle course management system. 
Experts in special education and educational technology were interviewed and surveyed for their responses toward 
the system. The results showed that the e-portfolio system was accepted by the interviewees. 

Keywords: E-portfolio, Technology Integration 



Introduction 

Traditional pencil and paper assessment in student achievement is to test students’ understanding toward the 
learning content within limited duration. Especially for students with special needs in learning, the pencil and paper 
assessment is not suitable because these students may have difficulties taking the tests and the results may not reflect 
student achievements qualitatively. As a result, it called for an integration of various information resources in order 
to demonstrate learning outcomes from students with special needs. Learning portfolios are often used to facilitate 
and present students’ learning outcome in the fields of special education. A student-centered “portfolio assessment” 
is commonly used to support special educators in promoting students’ self-determination abilities, monitoring 
learning progresses, and improving course plans (Agran, Blanchard, Wehmeye & Hughes, 2002 ; Demchak & 

Greenfield, 2000 ; Ezell, Klein & Ezell-Powell, 1999 ; Salend, 1998). 

A learning portfolio is commonly composed of evidences toward student learning that are collected regularly based 
on predetermined learning goals. Different evidences of learning such as test results, artifacts, reflections, and 
student feedbacks are collected in a learning portfolio to represent the events of learning as students proceed in 
learning (Arter & Spandel, 1992; Stiggins, 1997). However, the content of a traditional portfolio expends and 
requires more storage space as various printed documents are collected as time goes by. In addition, printout records 
and static pictures are not able to present student learning in a dynamic manner that is important to show student 
achievements in particular motor skills. Meanwhile, managing and updating student artifacts in traditional portfolio 
can be another issue that hinders the use of traditional learning portfolios. 

With the improvement of multimedia and Internet technologies, different formats of media such as digital picture 
and video can be utilized to represent artifacts of student learning. By integrating digitized artifacts, electronic 
learning portfolios have become easier to manage and updated and therefore draw more attention from educators 
and researchers than traditional learning portfolios. However, students with special learning needs may not be able 
to benefit from a “one size fits all” e-portfolio because each student needs to attain their individualized learning 
goals. This project intended to construct an e-portfolio system for special education by modifying open-source 
Moodle™ course management system. Users such as special education teachers and parents were surveyed and 
interviewed to achieve the usability of the e-portfolio system. 
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The development of the e-portfolio system 



The structure of the system 

In order to meet special education teachers’ and students’ needs, the construction of the system focused on the 
components for building learning portfolios and assessment of student artifacts rather than the built-in management 
features. The following components were developed for the prototype: (1) individualized learning objective system, 
(2) assessment and assessment guidance, (3) manage and input of artifacts, (4) guidance for using this system, and 
(5) indexing/linkage among objectives, artifacts, and assessment. The structure was determined after discussions 
with the teachers, parents, and educational technology experts. 

The features of the system 

There are two major functions of in the prototype system, namely the user and the administrative functions. The 
components mentioned earlier are work for the administrative function. Learning artifacts can be presented 
dynamically in either html or Microsoft Word. These standardized digital formats provide convenience for viewing 
the progress of student learning and are also easier for later editing works. Meanwhile, the html content is organized 
as weblogs of the class and of individual students to give teachers and parents a clear view regarding the student’s 
learning progress day by day. 

The teacher will be able to view all the students in the teacher’s class while each parent can only view the portfolio 
of her/his child. The teachers were able to login to the administrative section to input and manage student artifacts, 
select learning objectives from a pull down menu, and key in their comments. The same learning objectives can be 
assigned to different students in the teacher’s class. Each student can also have different combinations of learning 
objectives based on the progress of the student and teacher’s decisions. Student artifacts can be stored in text, static 
image, and video formats. These artifacts are linked and indexed to the pre-determined objectives. Each learning 
event can be indexed to multiple learning objectives and vice versa. In the assessment section of the system, events 
of student learning are dated and listed with hyperlink functions. Each link will lead to a web page showing artifacts 
and related objectives of particular learning events accompanied with grading options and textbox for feedback input. 
The teacher can then evaluate student learning on the summarized page. An overview page on which digitized 
artifacts, objectives, and comments from the teacher can be generated automatically to summarize each student’s 
learning progress and achievements. 



Evaluation of usability 

Four special education/educational technology experts, four special education teachers, and four parents of students 
with intellectual disabilities were interviewed to evaluate the usability of the system. Questions focused on different 
perspectives such as the interface design and functions, the ease of use and management, and the content from 
output web pages were given to the educational technology experts, the teachers, and the parents respectively, to 
attain the usability of system. The following findings are summarized from the collected comments: 

1 . The system provides required features for building portfolios electronically. Student portfolios become dynamic 
feature by presenting learning events in multiple formats of information. Presenting students’ learning 
dynamically provides opportunities for efficient communication between the teacher and the parents. 

2. Indexing and linkage of the learning objectives, artifacts, and assessment criteria help individualizing learning 
goals and achievements, which are important for students with different intellectual disabilities. Meanwhile, 
matching each special education students’ achievement with individualized learning goals and artifacts help 
teachers to make plans for on-going teaching tasks. 

3. The output of the e-portfolio is easy to view and edit, which can be beneficial to promote the implementation of 
portfolio assessment. 

4. The dated digital artifacts with matching objectives not only give the parents easy access to the students’ 
performance in the classrooms but also help the parents to realize their children’s learning progress clearly. 
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Short Description 



Numerous organizations realize that continuous improvement in employee job skills and a business’ 
capacity to expand its learning is essential in today’s global competition. An ideal of learning organization 
was created to meet these needs of the corporate world. Team learning connects individual and 
organizational learning in such an institute. This literary review aims to identity barriers to team learning, 
possible means to overcome them and successfully foster virtual teams in a learning organization. 



Introduction 

To remain competitive an organization must renew its knowledge base and shift its paradigm constantly. A 
learning organization is supposed to be able to develop groups of competent employees, to produce optimal 
performance, to deal with the rapid changes in the market and ultimately accomplish superior economic 
achievement for the organization. Team learning provides the bridge between individual and organizational learning 
in an organization like this. However, people express concern that if team learning can really happen or be fostered 
due to the hierarchical bureaucracy and the prevailing culture of individualism that exists in most organizations. 



Definition of a Learning Organization 

A learning organization is defined by Peter Senge (1994) as an institute where people continuously increase 
their ability to produce desirable results, their new ideas are open-mindedly encouraged and they have the liberty to 
learn and achieve together. A learning organization can be beneficial in two different perspectives: for the 
organization as well as for the employees. 

For the organization, it can increase competitive advantage because employees can learn faster than its 
rivals (Senge, 1994; Tobin, 1993); respond to change promptly because people know how to anticipate changes and 
how to create the kinds of changes they want (Senge, 1994); ensure superior performance through development of 
individual employees (Tobin, 1993). 

For the employees, it can cultivate mastery of their job skills as well as self-determination and learning 
strategies; improve quality of work life because employees are in an environment that is open for critiques and 
recommendations; enhances people’s awareness of collaboration and interdependence (Senge, 1994). 



Barriers to Team Learning 

Organization structures may either help or hinder learning. They affect team learning and put up barriers 
through (Marquardt & Reynolds, 1994): a) bureaucracy; b) poor communication; c) poor leadership; d) resource 
utilization or the lack of it; and the e) huge size of the organization. 
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Team learning is also the most challenging to many people and organizations. The process of learning how 
to learn collectively is unfamiliar. Most of us have had no training in this. There will be times of frustration and 
perhaps embarrassment, as the team members develop their collective capabilities. 



How to Overcome Barriers and Foster Virtual Teams 

A virtual team is a group of people, who might reside in various locations or cultures, involving in a 
technology-enabled team for the purpose of accomplishing learning or job tasks (Alavi & Yoo, 1997; Desanctis & 
Poole, 1997; Jarvenpaa & Leidner, 1999). 

More and more virtual teams have been used in organizations and higher education as a result of the 
development of technologies and globalization. They have become an essential process for team learning as well as 
task completion. 

Team Process 

To become an effective self-managed learning team requires a group of people to go through four stages of 
development , in each stage learning and practicing new skills. These stages are (Tobin, 1993): a) Formation, b) 
Development of team skills, c) Development of management skills, and d) Self-management. The result will be a 
team that not only manages its work, but also itself. 

Virtual teams also go through these stages, but the process moves much faster than in face-to-face teams 
(Johnson, Suriya, Yoon, Berrett & La Fleur, 2002). This is why using team-building exercises to encourage 
information exchange, to build trust and establish team identity can be valuable in facilitating this fast process 
(Jarvenpaa, Knoll, & Leidner, 1998). If the design of team interaction entails goal setting and strategic planning for 
completing learning or tasks, it is more likely for participants to achieve “shared mental models” (Powell, Piccoli & 
Ives, 2004). 

To ensure the success of a virtual team process, managers, administrators or instructors might also want to 
reflect on participants’ “psychological profile” and “personality characteristics” during the design of the team. 

Some of the traditionally honorable features, such as patience, persistence, perseverance as well as tolerance, 
flexibility and understanding, are also considered virtues for successful team players in this technology-enabled 
setting (Warkentin, Sayee & Hightower, 1997). These procedures might help mitigate the negative effects caused by 
ideological barriers. 

Communication 

The most effective practice suggested by Senge (1994) for team learning emerges from two conversational 
forms: dialogue and skillful discussion. We need to promote and provide opportunity for these forms of conversation 
among team members or the whole organization to facilitate team learning. 

Similarly, Jarvenpaa, Knoll, & Leidner (1998) concluded from their study that frequent and predictable 
conversation and regular feedback improve communication effectiveness, and eventually brought about higher trust 
and virtual team performance. 

In addition, to improve communication and collaboration within the virtual team, Warkentin, Sayee & 
Hightower (1997) suggested utilizing technology-enabled informal face-to-face contact early in the process; and 
Malhotra etal. (2001); Tan etal.(2000); & Wartkentin & Beranek, (1999) recommended developing a protocol for 
coordination and employing communication Training interventions. 

These measures might help alleviate the negative outcomes caused by: a) bureaucracy; b) poor 
communication; c) poor leadership; and the e) huge size of the organization. 



Technical Expertise 
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As far as technologies are concerned, virtual teams should be provided with sufficient knowledge searching 
or building gears and an invigorating environment for community development (Bitter-Rijpkema, Martens & 
Jochems, 2002). 

Virtual teams tend to have a variety of skill levels and sets, and may need to negotiate what specific 
technology to use for completing learning or job tasks (Sarker & Sahay, 2002). We need to encourage virtual teams 
to adapt technologies (Maznevski & Chudoba, 2001) because overcoming technical uncertainty and challenges 
might even bring out the development of trust and collective identity/capabilities (Javenpaa & Leidner, 1999). These 
suggestions might help moderate the negative consequences caused by the lack of resources. 

Conclusion 

Nowadays, to a large extent, team learning experience has been transferred to an online format. 

Virtual teams can be one of the building blocks (Powell, Piccoli & Ives, 2004) if a company or higher education 
institute would make the commitment to become a successful learning organization. The above-mentioned measures 
might help foster virtual teams and build a foundation for them to thrive in these organizations. 
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Abstract: Starting in the summer of 2004, the department of Curriculum and Instruction at Iowa State University 
began offering an online Master of Education (M.Ed.) with a specialization in Curriculum and Instructional 
Technology degree program. All of the required courses were delivered at a distance using a variety of technologies. 
However, students traveled to campus for a face-to-face session at least once per semester. The purpose of this paper 
was to provide a detailed curriculum analysis of this degree program using Posner’s (2004) curriculum analysis 
questions: curriculum documentation, curriculum proper, and curriculum in use. We employed a document analysis 
method, and examined an article published by the curriculum designers, course syllabi, the program website, and 
recruitment brochures; and interviewed key participants in the development of the program. Based on our analysis, 
suggestions were made to improve the curriculum and overcome potential weaknesses such as offering students a 
more diverse and dynamic program of study based on student interests, and providing instructors with training and 
resources necessary to create successful online learning environments. 



Descriptors: Online Education, Curriculum Analysis 

Introduction 

Starting in the summer of 2004, the department of Curriculum and Instruction at Iowa State University has 
been offering an online Master of Education (M.Ed.) with a specialization in Curriculum and Instructional 
Technology degree program. The purpose of this paper was to provide a detailed curriculum analysis of this degree 
program. Therefore, to identify the scope and focus of the curriculum, a systematic analysis was employed using 
Posner’s (2004) curriculum analysis framework, which poses four sets of questions that let “the analyst tear a 
curriculum apart into its components” (Posner, 2004, p. 1 8). These questions address curriculum documentation and 
origins', curriculum proper, curriculum in use and curriculum critique. Although the analysis was peculiar to this 
specific program, the results may be pertinent to similar programs and may provide a baseline for curriculum 
developers. 

The analysis is provided in two phases. First, the curriculum of the degree program is presented so that the 
reader can gain a better understanding of how the program was designed and implemented. Second, the curriculum 
is examined in the light of Posner’s curriculum analysis framework in order to identify the strengths and weaknesses 
and so that suggestions for improvement could be made. 
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Curriculum Overview 



As stated in Correia et. al. (2009) the online program was designed in 2003 by the faculty members 
teaching the same M.Ed. program in a face-to-face format. The reason for creating an online program that mirrored 
the face-to-face version was to meet the needs of full time K-12 teachers who are spread across rural areas of Iowa 
and are not able to attend classes in a traditional university setting (Michelini & Yu, 2009). 

Using a cohort approach, all students attend required courses at a distance using a variety of synchronous 
and asynchronous tools. However, students travel to campus for a face-to-face session at least once per semester. 

At the program level, a vertical organization is used that divides the program timeline into semesters via 
academic years. A total of 32 credits are required for successful completion of the 3-year program. The sequence of 
courses is primarily fixed with the exception of 3 hours of elective credits preferably taken after the first year. The 
program timeline (Table 1) appears to be layered and progressive with introductory/foundational content placed 
early in the timeline and more specialized content (i.e. theory and research) appearing later. The organization of the 
program also requires a capstone course at the end of the third year of the program requiring students to develop a 
portfolio (Cl 599B) that includes compiling both an artifact collection and writing a reflection paper. 



Table L Program Timeline 



Year 


Summer 


Fall 


Spring 


1 


Cl 615B - Instructional 
Technology Seminar (1 
credit) 

Cl 501 - Foundations of 
Instructional Technology (3 
credits) 


C I 505 - Introduction to 
Using Technology in 
Learning and Teaching (3 
credits) 


C I 507 - Principles and 
Practices of Distance 
Education (3 credits) 


2 


C I 594 - Contemporary 
Curriculum Theory and 
Principles (3 credits) 


C I 503 - Designing 
Effective Learning 
Environments (3 credits) 


C I 515 - Action Research in 
Education (3 credits) 


3 


C I 511 - Technology 
Diffusion, Leadership and 
Change (3 credits) 


HPC 588 - History of 
American Education (3 
credits) 


C I 599B - Research 
Development Project (3 
credits) 


Additional course work (3 credits) can be taken any time, preferably after the first year. 


Total Credits: 32 







Curriculum Analysis 

We employed a document analysis method which included an article published by the curriculum 
designers, course syllabi, the program website, and recruitment brochures; as well as an interview with one key 
participant in the development of the curriculum. Based on the analysis, suggestions were made to improve the 
curriculum and overcome potential weaknesses such as offering students a more diverse and dynamic program of 
study based on student interests, and providing instructors with training and resources necessary to create successful 
online learning environments. The following is a detailed curriculum analysis of this degree program’s course of 
study using Posner’s (2004) Curriculum Analysis Framework. 



Curriculum Documentation and Origins 

Curriculum documentation and origins address issues of who develops the curriculum for what reasons, 
how it is planned out, and how it is affected by social, political, cultural and or economic situations (Posner, 2004). 
In other words, the curriculum analysis starts with revealing the story behind its development. 

We based our analysis of this curriculum on five different types of resources. First, we reviewed the 
program’s website (http://ctlt.iastate.edu/~citmed/) to identify the purpose of the program, target audience, course 
schedule, tuition, and admission requirements. Second, we consulted informational and marketing materials that are 
used to advertise the program and present the guidelines for the creative component project that students are 
expected to complete at the end of their program of study. Third, we reviewed an article written by the curriculum 
developers that provides an overview of the program and a closer look at the courses each cohort of students are 
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required to take. Fourth, we consulted the Iowa State Course catalogue to gain any additional information about the 
required courses and their placement within their respective colleges. Fifth, we interviewed the director of the 
Center for Technology in Learning and Teaching who is one of the lead designers of this curriculum in order to seek 
any additional undocumented information that would be of relevance for this analysis. 

The manner in which the available documents (i.e. website, article) were structured indicated an emphasis 
on the program of study, with each explaining the courses required for the degree and in what order they need to be 
taken. As a result, as Posner (2004) states, the developers most likely viewed this curriculum "as a series of courses 
that the students must get through" (p. 11). It is then assumed that after students have progressed through the 
required courses in a pre-determined order, they will be able to use technology as teachers and leaders in their 
educational contexts. 

The biggest limitation in terms of curriculum documentation was a lack of archived “official” 
documentation that could be consulted in order to gain a more in-depth analysis of the curriculum. Having to 
compile different types of materials that provided information in bits and pieces and draw conclusions based on this 
limited information was not as robust, as preferred for this analysis. Though the program website provided the most 
details about this curriculum, its target audience is prospective students who are most likely already interested in 
pursuing an online degree thereby the information it provides is somewhat narrowly focused. It would have been 
helpful to have access to documentation that stated the goals of the program, as well as any specific objectives 
outlined during the design process. Regardless, in this current program of study students are required to take pre- 
defined courses in a pre-determined order, but are not necessarily told why these courses were selected over others 
particularly since the on-campus format offers students some flexibility with course selection. 

In the curriculum development process, Schwab (1971) suggests that the learners, the teachers, the subject 
matter and the milieu should be represented, as they are the four commonplaces of education. The developers of this 
current curriculum served multiple roles as teachers, subject matter experts, and educational psychologists each 
raising the concerns shared by Schwab (1971) in terms of over or under representation of each commonplace. 

It can be argued that the learner was represented by the faculty members who were directly involved in the 
development process, with each possessing an extensive background in teacher education, as well as learning theory 
and educational psychology. Therefore, it can be assumed that their understanding of students, how they learn, and 
what their needs are, were likely represented. 

Teachers are likely to be the best-represented group within this curriculum development team as the 
developers are also the ones teaching in the online program itself. Thus, they can understand the complexities of 
online learning, as well as the demands and constraints that might be faced during the implementation of this 
curriculum. Also, as experts and practitioners in the field of Instructional Technology, the developers have a strong 
theoretical background in how students learn with technology and what types of practices are required that may not 
be particularly an issue in a face-to-face classroom environment. However, not all required courses are taught by 
Curriculum and Instructional Technology faculty, who are already presumably interested in and qualified to take 
part in distance education and technology initiatives and integration, therefore the level of experience and expertise 
with online learning environments amongst the other instructors outside of the department vary a great deal. 

Students are also required to take several courses that may not have been originally designed or taught 
online before. Therefore, faculty members may be asked to teach their course online primarily to best-serve the 
program, each of whom may not be experienced or trained to teach in this type of learning environment. As a result, 
some faculty members might have little theoretical and practical understanding of online learning principles, 
however should have been represented in the curriculum design process in order to better serve the incoming 
students of the program. The curriculum designers likely tried to fill this gap with the hiring and implementation of 
a distance education coordinator, whose purpose is to assist faculty with the development of materials and 
organization of their courses for this online program. 

The team members also acted as the subject matter experts in the design of this curriculum. As the 
practitioners of the curriculum, faculty members have the theoretical and practical background about what teachers 
should know in order to integrate technology in a way that will improve student learning. For example, students are 
expected to take classes outside of their own specializations, offering them an opportunity to better conceptualize 
learning theories and develop professionally. 

One goal this curriculum claims to meet is offering full time K-12 educators who are spread across the rural 
areas of Iowa and unable to attend traditional university classrooms an opportunity to pursue a continued and/or 
graduate education. Nevertheless, this program is not limited to Iowa teachers; anyone who meets the admission 
requirements may apply to the program. As a result, no one from the community it is intended to serve is 
represented in the curriculum development process. The curriculum developers analyzed the current educational 
landscape and identified a need for the program based on their own analysis. For example, Coreia et. al. (2009) 



155 




reported that the state of Iowa had a rural population of 1,337,000 out of a total 2,982,000 people. This represents a 
relatively high percentage of the total population and presents a potential opportunity for reaching out to both 
teachers and schools in remote locations. 

The educational, social and political context at the time of the development of this online M.Ed. program 
appeared to value the role of technology in schools, and the necessity for preparing teachers with the new 21 st 
century skills that they could then foster unto their own students. For instance, the 1997 National Council for 
Accreditation of Teacher Education (NCATE) reported, "[t]he nation's teacher education institutions must close the 
teaching and learning technology gap between where we are and where we need to be. Teacher education 
institutions must prepare their students to teach in tomorrow's classrooms" (p.ii). Since then there has been a 
growing expectation for technology integration in all levels of education, and teacher education programs have been 
considered to be the best and most practical places to educate pre- and in-service teachers on meaningful technology 
integration practices so they can foster student learning in their own classrooms. 

Yet, thus far single educational technology courses have been the most common practice for technology 
preparation in pre-service teacher education (Hargrave & Hsu, 2000). This single stand-alone course approach may 
not be as effective for students in learning about what technologies they can use in their particular content areas, as it 
typically only provides generalized overviews of a range of technology tools (Mehlinger & Powers, 2002). This is 
also a reason for why the curriculum developers identified a need for more opportunities in Iowa for graduate 
education in the field of educational technology. 

In regards to the design and development of the curriculum elements such as content, characteristics of 
audience, activities, materials, sequencing principles and evaluation, each are represented to some extent. However, 
the weight of each planning element was not equal in terms of the information provided by the various curriculum 
materials analyzed. For instance, the program website emphasizes the list and the sequence of the courses that need 
to be taken, which also alludes to the focus of the content (or the subject matter). The common instructional theme 
among all the courses is the extensive use of discussion boards, which are intended to supplant the traditional 
classroom discussion. These types of forums have both pitfalls (e.g. lack of nonverbal cues) and benefits (e.g. 
everyone has a voice, time for reflection before posting a response). In general, course materials (i.e. textbooks) are 
the same as what is being used in the traditional, face-to-face university setting. Evaluation varies with each course, 
however the most common practice appears to measure learning through project-based activities that require 
students to apply what they have learned into practical classroom situations. 



The Curriculum Proper 

The curriculum proper is concerned with the purpose and content of the curriculum and what assumptions 
each imply (Posner, 2004). Our analysis indicated that an assumed overarching goal of this program is to encourage 
students to use technology in their own educational situations or contexts so that they can foster learning and 
provide opportunities for their students to develop skill sets that will help them be successful in the future. After 
students have completed the required course work and creative component, it is assumed that they will be prepared 
to apply many of the technology integration concepts learned into their own classrooms. 

With limited documentation, it is very difficult to identify an explicitly stated purpose associated with the 
curriculum. The only statement that can be regarded as a potential purpose statement is provided on the program 
website. It states "The Master of Education in Curriculum and Instructional Technology program is designed for 
teachers and other educators to engage in professional development in their knowledge of subject matter, teaching, 
and technology by incorporating contemporary tools and resources" (Michelini & Yu, 2009). Despite its brevity, this 
statement encapsulates all four categories (i.e. personal development, socialization, economic productivity, further 
learning) of educational aims (Posner, 2004). 

Professional development is directly related to personal development where students will hopefully 
improve both their technical skills (training) and effective technology integration (education). Correia et. al. (2009) 
state that students are grouped into cohorts in order to create a supportive and collegial environment that is directly 
related to the socialization aspect of education. However, it also should be noted that socialization in an online 
environment is quite different than face-to-face environments. The physical distance between the students and the 
instructor can hinder effective communication since it is essentially dependent on a different, and less-used skill 
sets. This is why the continuity of a cohort format is considered to be so important in online learning (Correia et. al., 
2009). 

Economic productivity is primarily related to acquiring job skills in the students target area. Although most 
students are already full time teachers, earning a Masters degree often provides them opportunities for advancement 
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and future career options. For example, one graduate stated, "While working toward my masters’ degree I applied 
and accepted my current job at the Area Education Agency ... my background and masters in technology is 
extremely valuable" (Correia et. ah, 2009). Finally, further learning is one of the basic aims of the program. 
Namely, it is impossible to teach all the available technologies that can be used in different content areas, or to guess 
what changes and improvements in technology will happen in coming years, instead, this curriculum offers students 
the theoretical framework that will help them explore what is available and what could be done in their own 
classrooms. 

Also, what type of learning objectives are included or emphasized in the curriculum might give an idea 
about the approach developers took in developing this program. In particular, the learning objectives are represented 
at the course-level through the individual course syllabi. Flowever, some are presented in a way that cannot be 
identified fairly easily as individual or personalized learning objectives. Namely, many learning objectives are 
tangled amongst course overviews and educational goals listed on course-related documentation. For example, three 
course syllabi (chosen based on availability) were analyzed to find what learning objectives are included and 
emphasized as representative of the curriculum. For example, the F1PC 588 course description (See Appendix A) 
presents just one overall learning objective after listing the topics that will be covered in the course. 

The Cl 511 course overview (See Appendix B) is also designed in a similar fashion to the F1PC syllabus. 
After briefly listing the content that will be covered, the instructor concludes with a sentence that summarizes what 
students will have learned at the end of the course. Flowever, in this example, there is also an emphasis on the 
application of skills (e.g. providing leadership) in relation to Bloom's (1956) taxonomy of application in the 
cognitive domain. Flowever, skills like leadership are often not gained by following a “recipe” type of instruction. 
Students should draw their own conclusions based on their conceptual knowledge acquired throughout the course, 
and their own attitudes towards how to be leaders in technology integration using their cognitive strategies (Gagne, 
1977). 

The Cl 505 course purpose and approach (See Appendix C), which utilizes a more application- 
based approach to instruction, also does not provide well-defined learning objectives. The approach that the 
instructor has taken also reflects a combination of intellectual and skills knowledge as students are expected 
to apply the conceptual knowledge they have gained and create classroom activities for their learners. 

Overall, we can conclude that the curriculum developers do not follow any specific educational psychology 
model (e.g. Bloom, Gagne, Ryle) in developing their objectives. Rather, they use a combination where they can 
emphasize what they consider important for their target students, based on each specific course. Each of the course 
descriptions or overviews appear to be exactly the same as their face-to-face equivalent, resulting in the instructors 
to continue aiming towards achieving the same educational objectives, despite the course being offered through a 
different medium. As a result, students complete similar assignments and learn similar material in almost the same 
amount of time regardless of the medium. 

The manner in which the subject matter is represented in the curriculum indicates more of a pedagogical, 
than behavioral focus. The required classes in the curriculum covers both theoretical foundations (e.g. how people 
learn) and practical applications (e.g. designing an instructional unit), so it can be concluded that this curriculum 
conceives of Curriculum and Instructional Technology as a combination of theory and practice. In some courses, 
the focus is more on theory (e.g. F1PC) while in others there is more of a balance between theory and application. 
For example, in Cl 505 after students learn how and why wikis can be used in educational settings, they design a 
learning activity using this specific technology. This combination of theory and practice indicates a pedagogical 
agenda that technology needs to be used as an aid to foster student learning rather than being a controlling factor. 
This emphasizes the point that, in none of the classes offered throughout the program will you find learning about a 
specific technology taught as an isolated skill. Students are encouraged to explore their options and develop 
technical skills by working on technology integrated projects, which ideally accomplishes the primary goal of 
creating similar environments for their own students. 

Most content is usually presented through forum discussions and project-based assignments. In all courses, 
students discuss assigned readings in an asynchronous forum discussion, and complete assignments to show their 
understanding of the material. Flence, using this approach the instructor takes on a secondary role in the learning 
environment; however, this does not mean that they do not have influence on what is being learned. Instructors also 
participate in the discussions, and assist students in co-constructing knowledge, yet the format can limit their ability 
to interfere when they see a problem in student learning. Flowever, some topics may require explicit teacher 
lecturing, which can be problematic in online environments. As a solution to this problem, one instructor records 
PowerPoint presentations with audio tracks and uploads them to a course management system (Correia et. ah, 2009). 
Flowever, thought this may seem like a plausible solution, it still lacks student-to-student/student-to-teacher 
interaction that a face-to-face classroom provides which is a key aspect to learning. Likewise, an online presence 
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does have different dynamics than a physical presence, and can lack some very important elements necessary for 
student learning. 

With regards to a multicultural view, it can be argued that this curriculum does not represent a multitude of 
cultures. Most students in the program are practicing teachers across Iowa, which consequently does not represent a 
multicultural population. There are almost an equal number of males and females in the program, yet they are all 
typically Iowans. Additionally, very rarely do international on-campus students take these online classes. 

Despite the limitations mentioned, multicultural views are represented in the curriculum in two different 
ways: (1) how the faculty members accommodate the diversity of a student population; and (2) how they help their 
students form a multiculturalist perspective. To accommodate their students, faculty members encourage the 
inclusion of views that represent diversity, which incorporates Bank's (1988) idea of multiethnic education. For 
instance, when discussing the history of education, international students are encouraged to bring in examples from 
their educational systems and share experiences so that their classmates can have a better conceptualization of other 
cultures and educational systems. 

Another factor impacting the content is technology. Recent developments in technology have provided 
opportunities for more dynamic online classrooms where students and teachers are able to interact with one another 
more than ever before. A course management system, like WebCT, provides a repository platform where students 
and teachers can upload assignments and materials, as well as develop a virtual space for an online community. Web 
2.0 tools (e.g. blogs, wikis, social networking) provide user-friendly environments for students to collaborate, reflect 
and co-construct knowledge. Computer mediated communication tools (e.g. chat, video conferencing) help students 
and teachers at a distance to have virtual office hours where they can discuss issues and problems. Digital 
production tools (e.g. video and audio) provide a solution to the lack of face-to-face communication, and provide 
opportunities for students and teachers to have a virtual presence in an online environment. Using these different 
types of technologies, faculty members also model what students could do in their own classrooms, as well as 
enhancing their own classroom communication. 

Online education also incorporates some assumptions that underlie the curriculum approach to content and 
purpose of the program. In distance learning environments, the responsibility for learning rests more on the students 
as they are expected to be self-directed learners who can manage their time effectively and actively participate in the 
learning process. As stated earlier, the content delivery is mostly structured around forum discussions where 
students reflect on readings, comment on each other's ideas, and participate in a collaborative learning environment 
(Correia, et. al. 2009). In such environments, as Resnick (1983) suggests, rather than passively receiving 
information, students construct ideas and generate meaning through engaging in reflective discussions. Since most 
students are practicing teachers, they have preconceptions about their profession and are expected to make 
connections about what they learn in the courses (in terms of both theory and practice) and their own teaching 
experiences. 

Social interaction is another important element central to learning (Resnick, 1983) via this curriculum. It 
provides opportunities for students to work collaboratively, engage in constructive criticism, as well as reflect on 
and revise their own ideas (Posner, 2004). In online classes, most students are aware that their peers will see their 
contributions, so it is highly likely that they will pay more attention to detail regarding what information they share 
and how. Also, as required in most courses, they are expected to comment on their peer’s entries and ultimately 
build towards creating a structured, co-constructed knowledge base. 

Overall, it might be concluded that a constructivist perspective seems to dominate how learning and 
teaching is viewed in this program, as well as how activities are structured. However, rather than a conceptual 
change model, it is closer to a cognitive apprenticeship model where students engage in real life tasks in a 
contextualized manner with on-demand help by instructors. 

In addition to content learning, online learning contributes students’ self-development, as it requires 
students to be self-directed, motivated, and effective time managers. Though they do not have to commit to a 
regularly scheduled time and place for class, they do have to participate and contribute in other meaningful ways. 
Being a part of an online community, being able to effectively manage their time, being responsible for their own 
learning (through readings, discussions, projects), as well as ethical (i.e. cheating) and professional guidelines are all 
aspects of a hidden curriculum that are not directly addressed in the “official” curriculum. 

A common implication related to online learning is the perceived lower quality of education that students 
receive as a result of being associated with an online degree program. As part of an effort to deemphasize this 
perception, developers focus on the relatively high retention and graduation rates achieved by the program. Several 
students of the program also suggest that as a result of good online course design it has allowed them to be 
successful (Correia, et. al, 2009): 
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“I have found this to be much more engaging and meaningful than meeting in person in most of 
my undergrad courses. I really like being able to participate in a discussion, but having the time to 
really think through things before ‘opening’ my mouth, so to speak. Also, there is almost more 
accountability with a course like this than with a traditional course.” 



The Curriculum In Use 

The curriculum in use is related to how teachers interpret and put the official curriculum into operation. 
Throughout this implementation process, they need take many frame factors —physical, cultural, temporal, 
economic, organizational, political-legal, and personal characteristics— into consideration. These factors might either 
hinder or facilitate the implementation of the curriculum (Posner, 2004). 

In terms of temporal factors, the flexible nature of this online program essentially makes each course a 24/7 
commitment for a student to be engaged each semester. There is less structure around most asynchronous 
communication formats with the exception of deadlines. Each course is part of a full-time professor’s teaching load 
and is delivered based on his or her own design and preference. 

Physical factors play a different role in online teaching environments. Although they do not require walls 
and chairs, other tools that would facilitate the learning experience are needed. In this particular curriculum, with 
the exception of a brief face-to-face meeting associated with some courses, the commitment of any indoor or 
outdoor facilities is minimal. However, the technological requirements for participating in online courses is much 
more demanding. It is expected that each student has access to the required equipment associated with each course, 
which are often already available at most public libraries and universities. It is assumed that the department is 
already fully equipped to administer this program and students must be prepared for or gain access to the necessary 
and compatible equipment that may extend beyond a desktop or laptop computer (i.e. webcam, microphone, video 
conference software/site, etc.) 

Regarding the organizational frames, this program does employ a full-time distance education coordinator 
that serves as a technical support specialist and provides assistance with the acquisition of any specific technical 
requirements associated with each course. The nature of this program requires a wide range of technologies as both 
delivery mechanisms and subject matter. A student’s ability to seek resources and assistance, from both inside and 
outside of the university and department when necessary is critical to their success. 

Political-legal factors refer to the impact of outside organizations have on the curriculum. For the particular 
curriculum under analysis, in addition to the accreditation offered by the department. University and state, the 
courses associated with this program are also affiliated with the traditional M.Ed. graduate program. A limited 
number of students not associated with this program are able to enroll and participate in these courses for credit 
towards their own respective programs. These courses are met with the same rigor and valued the same as all other 
graduate level courses offered. As a result, the associated costs with the implementation of this curriculum range a 
great deal. First, a staff position (i.e. distance education coordinator) is dedicated to supporting this program. 
Second, faculty planning and preparation time will vary from traditional implementations of the same course. Third, 
technical resources must be updated (if necessary) to accommodate the distance education initiatives that each 
course demands. 

In terms of consistency with the teachers’ attitudes, beliefs and competencies this curriculum has the 
advantage of instructors being the developers as well. Most faculty associated with the program have had 
experience teaching in both traditional and online environments; as well as worked with various technologies in the 
past, utilizing them to the best of their abilities. However, no formal training associated with online learning and 
course development is offered to faculty in their preparation for teaching in this program. The staff development 
necessary to support this program is focused around the distance education coordinator’s role, as well as all other 
university-wide support services. The expectation that all faculty members are of equal technical ability is 
unrealistic. However, it is assumed that they will seek and receive the appropriate assistance necessary to maintain 
the effectiveness of their course design and delivery. 

Most technologies are already available and accessible to both faculty and students. It is expected that 
students who enroll in distance education programs have access to some level of technology appropriate for 
completing the associated coursework. Often a course management software solution is complemented with other 
technologies used at the discretion of the faculty member. 

Mastery of specific technologies are not required to meet the expectations of this program, instead 
representing how technology can be used to enhance learning is valued. This is expected through the demonstration 
of leadership, competence, creativity, and understanding of a particular subject matter. Based on how each course is 
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designed, the proper resources and assistance must then be coordinated. The role of the distance education 
coordinator or the utilization of other university-wide resources is critical in meeting this need. 

When considering students’ cultural, ethnic or social backgrounds, this program assumes that each student 
has the ability to effectively participate in an online learning environment. First, students are expected to be able to 
overcome or accommodate any language or communication barriers associated with an online learning format. This 
can be especially difficult when both visual and non-visual cues are often relied upon when communicating in order 
to decode or interpret what and how information is being shared (i.e. the use of slang or acronyms). Second, the 
practice of “netiquette” is an important component to online learning that all students should consider. Often, 
common communication characteristics such as tone, expression, physical gestures, etc. are lost in online learning 
environments which require careful scripting of how and in what manner students share and express their thoughts 
and opinions. Third, in order to participate in online learning environments certain levels of economical and 
technical resources are assumed. Though ownership of the various technical aspects required may not be necessary, 
students are expected to have reliable, consistent and frequent access to the tools necessary to fully participate and 
contribute in the courses. The value placed on repeated student-to-student/student-to-teacher interactions is great 
and essential to each course’s success. 

Though this curriculum remains a fairly new and popular program of study, several collaborative 
approaches are a part of its continued modification efforts. First, with the direction of curriculum change being 
handled by faculty members within the program, the potential for biased or discrete influences remains a possibility. 
Flowever, the curricular focus appears to rest around meeting each instructor’s teaching and learning beliefs. 

Second, the necessity for teacher growth to occur “by doing” and learning from their peers is also a factor 
in how courses are taught. The nature of online learning will most likely impact what courses are offered within this 
program, as well as who is best qualified to teach them with respect to both the content area and online learning 
principles. Any course a faculty member teaches within this program is included as part of their assigned teaching 
load with the expectation that they invest the time necessary to offer a successful course. 

Third, the ethnographic approach to how students are evaluated also influences the basic structure and 
change expectations for this program of study. The subjective nature of reflecting on and aligning program artifacts 
to a set of standards remains the primary benchmark to measuring the effectiveness of the program itself. The Cl 
599B portfolio course is purposefully placed at the end of a student’s program of study and designed as an outlet for 
them to express their own point of view of their total body of work and how it relates to their goals as a practitioner 
as suggested by Posner (2004). This reflection and demonstration is then evaluated by the faculty’s point of view 
and their interpretation of what academic growth and competency should look like after participating in this course 
of study. The reconciliation of the two viewpoints then allows for both faculty and students to validate their 
experiences in the program and measure their level of effectiveness. 

In terms of evaluation of the curriculum, there was little access to data related to this program with the 
exception of some student feedback, as well as retention and graduation rates. As a result of this minimal dataset, 
strong conclusions are difficult to draw. However, based on the two previous cohorts who participated in this 
program the following retention and graduation rates are as follows: 

Students beginning in the 2004 Cohort : 

Retention: 88% Persistence 

Graduation: 7 out of 8 receiving the Masters of Education degree 
Students beginning in the 2006 Cohort : 

Retention: 92% Persistence 

Graduation: 12 out of 13 receiving the Masters of Education degree 

The evaluation method used by this curriculum is likely to be a flexible and action-oriented approach. It 
advertises itself as being an applicable and adaptable graduate program that accounts for the flexibility and dynamic 
nature of the working professional. The emphasis placed on student-centered learning and relating coursework to 
individual situations makes it very difficult for a highly measurement-based evaluation to occur. 

One outcome concern we continue to have in relation to this curriculum is assessing student-learning gains, 
versus those who are enrolled in similar, but more traditional graduate programs. Obviously there are a variety of 
conditional factors that must be considered when attempting to adequately compare the two programs such as 
equality in student ability, prior knowledge/experiences and the influence a cohort format may have on learning. 
Ideally, this type of evaluation data could be used in comparing and analyzing how effective information is being 
shared and later applied by each respective group of students. Though retention and graduation percentages could 
potentially mark the success of a degree program, a true assessment of work quality and student competency may 
also be advantageous in verifying if an online learning program is truly beneficial as opposed to its face-to-face 
counterpart, or vice versa. 
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Critique and Suggested Revisions 

There are numerous strengths and weaknesses associated with this curriculum. First, its strengths include 
flexibility for serving a greater student population, utilizing a cohort format, and courses taught by full-time faculty. 

One of the greatest advantages of online learning and this program is its ability to provide educational 
opportunities to potential students who might not otherwise attend a traditional University setting for reasons such as 
geographical, career, family, etc. The flexibility to take classes no matter where you are located in the world during 
a time that is most convenient to you is a very appealing endeavor for a prospective student, as well as offers an 
entrepreneurial opportunity for a University to expand its student body. This curriculum is able to offer many of the 
same graduate-level courses, performance rigor and University accreditation as its face-to-face counterpart with the 
addition of providing students the option of being enrolled at a distance. 

The use of student cohorts is also a potential strength of this program. Since this curriculum is primarily 
evaluated based on retention and graduation rates, any attempt at enhancing these numbers should be considered an 
asset. Since most online learning environments can appear to be anonymous and lack a social connection, students 
are forced to find some type of relationship with a course that encourages them to stay engaged and actively 
participate. Taking advantage of this program’s highly structured program of study makes it possible for a group of 
students to enter and progress through as a single “class” thereby making it easier for students to make connections 
with their peers on a repeated basis over several semesters. However, just as this attempt at student continuity can 
provide a benefit to academic performance, the incestuous nature of the same experiences and ideas being shared 
over 3 academic years can also be detrimental. Recognizing this, several students who are not part of the program 
are allowed to enroll in each online course for the purpose of keeping the classes diverse and dynamic in what 
information is shared and discussed (Correia, et. al, 2009). 

Another strength of this program is having the opportunity to work with full-time faculty members who 
often teach both the face-to-face and online versions of the same course. Hence, online students are not placed at a 
disadvantage in terms of learning from experts in their respective fields of study. Overcoming the perception that 
instructors are less than adequate in online courses and that little interaction is ever had amongst students and faculty 
is critical in ensuring that students of this program are receiving the highest quality of education possible. As stated 
earlier, since this online program mirrors the face-to-face program and students often learn the very same things that 
on-campus students do, the program director stated that teaching online has “unofficially” made their face-to-face 
classes more effective (personal communication). Teaching the same course in two very diverse media forces the 
instructors to think about their courses differently, as well as how to communicate and employ various instructional 
techniques. This comprehensive look into their own pedagogy and course material has the potential to make 
instructors more aware of their own actions and identifying the most effective ways to illustrate or share 
information. 

Next, as part of this curriculum’s weaknesses and suggested revisions includes it’s rigid course structure 
(possible solution: provide students more course options) and lack of teacher training in online teaching (possible 
solution: offering training to all involved faculty). 

Though a student cohort system may work very well in an environment that students are already at a 
disadvantage in getting to know each other, and much of what makes so many online degree programs so popular is 
its flexible nature in completing courses from a distance the two are difficult to mix. An example of this inflexibility 
and limitation to being a part of this program is the restriction in what and when students can take various courses. 
All of the required courses for completing this program are offered once per academic year. This makes it very 
difficult for students to seek and substitute course alternatives or step away from the program for a period time (i.e. 
semesters) without penalty. Ironically, though an advantage of enrolling in an online degree program is flexibility, 
the nature of fixed course scheduling limits it a great deal. In the traditional, face-to-face M.Ed. program students 
are able to customize and choose courses with much greater ease. This is because of both the lack of schedule 
rigidity and cohort formatting. For example, in the face-to-face program students are given the option of choosing 
one of the several courses in the educational leadership area. However, in the online version they lack that option 
and are limited to only one course. Though each course was likely chosen on the basis of current student 
characteristics and needs of the target audience, these pre-defined course decisions limit a student’s freedom in 
designing their own program of study. 

Another weakness associated with this curriculum is the lack of training provided to faculty members 
teaching in online environments. Teaching online is neither easier, nor takes less time than face-to-face instruction. 
Rather, it requires careful planning and detailed course design that provokes and facilitates meaningful discussions, 
activities, projects. Regardless of the amount of teaching experience in traditional learning environments, 
developing the skills and characteristics associated online course development and instruction can be challenging 
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and time consuming. Hence, a training module for online teaching strategies offered to the faculty members 
involved with this curriculum is needed. Just as this curriculum prioritizes the success of its students, preparing and 
offering faculty members the resources necessary to develop and implement effective online instruction should also 
be highly valued. 



Conclusion 

As the phenomenon of online learning remains both a popular trend and challenging aspect of educational 
quality this curriculum has many promising characteristics moving forward as a result of this analysis. The priority 
now rests with overcoming potential weaknesses with possible solutions such as; offering students a more diverse 
and dynamic program of study based on their interests, as well as providing instructors the training and resources 
necessary to create just as successful online learning environments as face-to-face. Despite this program’s 
continued growth and perceived success, the challenges of online education remain to be ongoing. 
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Appendix A 

HPC 588 Course Description and Objectives 

This course explores the goals, purposes and problems of formal education, primarily at the 
elementary and secondary levels, in the United States from the colonial period to the present. We 
will utilize both primary and secondary sources to investigate topics including: varied localized 
approaches to education during the colonial period and the early republic: the movement fro 
common schools: the "feminization" of the teaching profession: the rise of the public high school: 
nineteenth-century forms of race and gender segregation; the growth of educational bureaucracy 
and vocational education; Depression and Cold War reforms; the desegregation struggle; and 
increasing federal-government involvement in education. Throughout, we will explore the 
connection between educational policies and larger social, political, economic and cultural 
developments. As a result, students will gain a better understanding of the origins and evolution of 
various current educational practices, policies, issues and dilemmas. (Fairchild, 2008). 

Appendix B 

Cl 511 Course Overview 

The purpose of this course is to explore the ways that information age technologies (broadly 
defined to include computers and related devices, as well as the Internet and other tools and 
applications) have been integrated into schooling contexts. We will begin by exploring a general 
model that addresses how innovations are diffused and adopted. We will then focus our attention 
on how educational change occurs. Finally, we will look at an example of how a particular 
innovation (use of computers) diffused into educational settings. Studying these perspectives will 
help us better understand how technology integration is occurring in each of our individual 
contexts, and to provide leadership to help make the integration process more efficient and 
effective. (Niederhauser, 2008). 

Appendix C 

Cl 505 Purpose 

The purpose of the course is to acquaint the student with current educational applications of digital 
technologies in teaching and connections of these applications to contemporary learning theory. 
Emphasis will be placed on both the "how" and "why" of using digital technology in classrooms. 
Integrating digital technology to improve instruction and create students centered classrooms will 
be the central theme of the course. In the course, students will be encouraged to connect current 
learning theory the creation of digital technology classroom activities for students (Thompson, 

2009). 

Cl 505 Approach 

This class will be structured using a combination of resources, discussion and student presentations. The 
course is project-based and almost every week, at the beginning of the semester, students will be asked to 
create a project (usually a student activity) around a particular topic. Each week, students will receive an 
agenda for the week and an assignment due at the beginning of the following week in class. The course will 
use a WebCT site that is used for communication, discussion, sharing resources, student grade information, 
and course information. In addition, we will use a course Facebook site for communications and 
teacher/student reflections (Thompson, 2009). 
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Abstract 

The goal of the project was for the graduate students to be able to develop, co-teach, and asses a series of 
lessons designed to enhance technology integration in the classroom. The technology selected must serve the 
learners in the learning process not be the focus of instruction. For each lesson, the following components were 
required: 1) related state and national technology standards; 2) assessment instillments; 3) detailed teaching plan; 4) 
instructional activities that integrated multiple learner-centered strategies; 5) evidence of use of problem-solving 
principles and classroom management techniques; and 6) a list of learning resources used. The graduate students 
elected to create lesson plans and associated activities that incorporated methods and strategies of problem-based 
learning, telecollaborative learning, integration of instructional tools (tutorials, software, games, and simulations), 
and uses of media and online tools from sources such as the World Wide Web. Graduate students selected teaching 
strategies that enable the technologies to be purposefully integrated into classroom instruction. In addition, the 
graduate students linked the lesson plans to the Illinois State Learning Standards (ISBE, n.d.), the National 
Educational Technology Standards for Students (ISTE, 2007), and the National Educational Technology Standards 
for Teacher (ISTE, 2008). Assessment tools were developed using RubiStar (4teachers.org) and the PBL checklist 
(4teachers.org). Specifically, assistive learning devices (ADL) were considered in order to serve the students with 
special needs. Findings and conclusions from the implementation and evaluation of the lesson plans will be detailed, 
as well as graduate students’ and instructor’s reflections on the experience and plans for future design and 
implementation. 



Project Background 

It is important that graduate students in the Department of Instructional Design and Technology at Western 
Illinois University gain real-world experiences in providing a supportive, technology-rich environment for K12 
student learning. The experience not only enriches the graduate students’ understanding of the technology 
integration and its application in the classroom, but also bridges these two parts as one sustainable process. 

Graduate students were enrolled in th e Integration of Computer Based Technology’ in Schools course. The 
course focuses on developing and practicing strategies and methods for integrating learning technologies into 
classroom and educational administrative environments. This practical approach guided the graduate students to 
select methods and teaching strategies that enable the technologies to be purposefully integrated into classroom 
instruction. Moreover, practices of what being written on paper in the form of lesson plan into actual situation would 
provide empirical evidences of what types of teaching methods and strategies that allow and enable technology to be 
integrated into classroom. Because of this, the development of innovative lesson plans through systematic phases is 
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foremost, not only to “open the door” for the technology to be integrated into classroom, but also to bridge the 
theories and the actual application of the selected teaching strategy and its technology. 

Learning activities designed in the lesson plan should support the student-centered approach through the 
implementation of learning strategies and the integration of technologies. To develop lesson plans that encourage the 
K-12 students to do interesting and fun tasks provide valuable real-world experiences for the graduate students that 
enrich their professional practices in the future. 



Project Goal and Objectives 

The goal of the project was for the graduate students to be able to develop, co-teach, and asses a series of 
lessons designed to enhance technology integration in K-12 settings classroom. 

In detail, the objectives of this project were: 

(1) To create lesson plans and associated activities that incorporate methods and strategies of problem 
solving, telecollaborative learning, instructional tools integration and uses of media and online tools 
for second grade students. 

(2) To co-teach three lesson plans designed to enhance the integration of technology into second grade 
classroom. 

(3) To assess the implementation of three lesson plans that incorporate methods and strategies that enable 
technology to be integrated into second grade classroom. 

(4) To reflect the experiences in implementing the three lessons and use the experiences as references for 
future development and implementation. 



Literature Review 

Learning Strategies and Technology Integration into Classroom 

The technology used in the classroom was not the focus of instruction, but it is a tool that serve the learners 
in the learning process (Thorsen, 2009). Using technology in this way often requires the teachers to change the way 
they teach as described by SEIRTEC (2001) in their lesson #5 about “Factors Influencing the Effective Use of 
Technology for Teaching and Learning” as follows: 

“We have found that when professional development and technical assistance start with a particular 
teaching or learning strategy that the teachers believe will benefit their students (e.g., project-based 
learning, cross-curricular thematic units, cooperative learning) and then help teachers discover ways 
technology is a tool that supports the strategy, teachers are usually eager to try both the new instructional 
strategy and the technology” (p. 1 4). 

Integrating technology into classroom requires the appropriate selection of learning strategies that enable and 
encourage the technology to be used. Thus, the teachers simultaneously and implement the instructional strategy, as 
well as use the technology that support the strategy. 

Some of the learning strategies that enable the technology to be effectively integrated into classroom are: 
problem solving, telecollaborative learning, integration of instructional tools (tutorials, software, games, and 
simulations), and uses of media and online tools from sources such as the World Wide Web. 

Problem-solving strategy (also called problem-based learning) is a learning strategy that guides the student 
to learn from solving the problem. Problem-solving strategy focuses on central concepts and principles of a 
discipline, involves students as active individuals in solving the problem, uses contextual problems, enables the 
students to construct their own knowledge, and produces products for their final projects (Thomas, Mergandoller, & 
Michaelson in Thorsen, 2009). The student then gain knowledge and skills by analyzing the problem, considering 
all possible factors, providing alternative of solution, and choosing the best solution to solve the problem. In the 
implementation of problem solving, reality-based and problem-centered materials are used and they often presented 
by media, such as written cases, videos, and computer-based situation. Students go to the school media center or 
using computer to access database or content from the experts via the Internet (Smaldino, Lowther, & Russell, 
2008). 

According to Thirteen Ed Online (2010), collaborative learning is “a method of teaching and learning in 
which students’ team work together to explore a significant question or create a meaningful project” (p. 1). 
Telecollaborative learning enables the students to collaborate with students or teachers in other places around the 
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world. In this case the role of technology, such as Internet makes the collaboration possible. The Thirteen Ed Online 
website emphasizes that collaboration can occur without regard to distance or time barriers by using the Internet. For 
example, e-mails can be sent to students or teachers around the world in order to share or exchange ideas or 
information. Skype application program can be used to have live conversation, chat and conference. 

The instructional tools, such as tutorials, software, games, and simulations, into the classroom can be 
effectively integrated into the classroom by managing the classroom activities for technology use. By incorporating 
learning materials into instructional tools above, provides powerful learning experiences for students (Smaldino, et 
al., 2008). In addition, the use of media and online tools enhance students’ learning especially to conduct research 
or gather information online. 

Innovative Lesson Plans 

Kizlik (n.d.) stated that “Lesson plans are written by teachers to help them structure the learning for 
themselves and for the students.” In addition. Brown and Green (2006) pointed out that lesson plans contain day-to- 
day activities that compose a unit. In the lesson plan, the activities are measured in hours or minutes. Innovative 
lesson plans that integrate technology through appropriate selection of learning strategies enhance students’ 
learning. The EdTechTeacher (2010) website strongly recommended the technology integration projects and 
activities to engender student creativity and empowerment through various innovative lesson plans. 



Methods 

For this project, the graduate students used a systematic phases in developing lesson plans for second grade 
students. The phases include analysis, design, development, implementation, evaluation, and reflection. 

Analysis 

According to Smith and Ragan (2005), there are three domains of instructional analysis: learner, task and 
context. In this phase, learners’ characteristics, learning content, and environment were analyzed. The learners’ 
characteristics of intended lesson plans were lesson plans for second graders at a private elementary school. All of 
them had basic skills in computer operation, including clicking, dragging, and listening instruction on how to use the 
navigation buttons. In addition, the learners have average to high cognitive abilities. Even though some of the 
learners did not have high performance in every subject, they mastered certain subjects greatly. Moreover, the 
children were emotionally stabile and socially friendly. 

The learning content that had been developed in innovative lesson plans were from the area of social 
science with topic “Native American Tribe”, the area of English language art with selected topic “Christmas around 
the World” and from the area of math with topic “Regrouping”. The content analyses of the topics were as follows: 

(1) Native American Tribes 

Native American Tribes was a topic in Social Science subject area. Through this lesson, the students learn 
about the Native American Tribes by listing the name of the tribes, locating on map where the different tribes lived, 
and describing one of the Native American Tribes. 

(2) Christmas Around the World 

As a topic in the area of English language art, Christmas around the World lesson allows learners to study 
how people around the world celebrate their Christmas included traditions (e.g., food, decorations, and rituals). The 
students also need to analyze the similarities and the differences between their traditions and other countries 
customs. 

(3) Regrouping 

The third topic was “Regrouping” in math. Though this lesson, the learning content provides the learners 
with chances to learn how to regroup ones as tens, count by tens and ones, and count two-digit addition to solve 
problems in their everyday lives. 

The analysis results of the learning environment show that the second grade classroom is located in very 
safe and convenient place to study. Moreover, the management and position of learning resources and media, 
especially the availability of computers and Internet accesses support the integration of various technologies in this 
classroom. 
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Design and Development 

The components of each lesson plan were: 1) related state and national technology standards, 2) assessment 
instmments, 3) detailed teaching plan, 4) instructional activities that integrated multiple learner-centered strategies, 
5) evidence of use of problem-solving principles and classroom management techniques, and 6) a list of learning 
resources used. The following information detailed the related state and national technology standards, the 
assessment instruments, the instructional strategies and activities, and technologies being integrated for each lesson 
plans. 

1) Related state and national technology standards 
Topic 1 : Native American Tribes 

The related state and national technology standards for this topic were: (1) the Illinois State Learning 
Standards State Goal 16 (Understand events, trends, individuals and movements shaping the history of Illinois, the 
United States and other nations) part A. la and part C.la and State Goal 17 (Understand world geography and the 
effects of geography on society with an emphasis on the United States) part A. lb; (2) the National Educational 
Technology > Standards for Students (standard 3c: evaluate and select information sources and digital tools based on 
the appropriateness to specific tasks and standard 5b: exhibit a positive attitude toward using technology that 
supports collaboration, learning, and productivity); and (3) the National Educational Technology > Standards for 
Teacher (standard Id: model collaborative knowledge construction by engaging in learning with students, 
colleagues, and others in face-to-face and virtual environments). 



Topic 2: Christmas Around the World 

For the Christmas around the world topic, the following state and technology standards were used: (1) the 
Illinois State Learning Standards State Goal 3 (Write to communicate for a variety of purposes) part B. la and part 
B.lb and State Goal 5 (Use the language arts to acquire, access, and communicate information) part A. la, A. lb, 

B. la, C.la, and part C.lb; (2) the National Educational Technology’ Standards for Students (standard 3b: locate, 
organize, analyze, evaluate, synthesize, and ethically use information from a variety of sources and media, standard 
3c: evaluate and select information sources and digital tools based on the appropriateness for specific tasks, standard 
3d: process data and report result, standard 5b: exhibit a positive attitude toward using technology that supports 
collaborative, learning, and productivity); and (3) the National Educational Technology’ Standards for Teacher 
(standard lb: engage students in exploring real-world issues and solving authentic problems sing digital tools and 
resources and standard lc: promote student reflection using collaborative tools to reveal and clarify students’ 
conceptual understanding and thinking, planning, and processes). 

Topic 3: Regrouping 

The related state and national technology standards for this topic were: (1) the Illinois State Learning 
Standards .State Goal 6 (Demonstrate and apply a knowledge and sense of numbers, including numeration and 
operations (addition, subtraction, multiplication, and division), patterns, ratios and proportions) part A. la and State 
Goal 8 (Use algebraic and analytical methods to identify and describe patterns and relationships in data, solve 
problems and predict results) part C.l; (2) the National Educational Technology’ Standards for Students (standard 
6b: select and use applications effectively and productively and standard 5b: exhibit a positive attitude toward using 
technology that supports collaboration, learning, and productivity); (3) the National Educational Technology’ 
Standards for Teacher (standard Id: Model collaborative knowledge construction by engaging in learning with 
students, colleagues, and others in face-to-face and virtual environments); and (4) the NCTM Standards and 
Expectations (number and Operations Standard for Grades Pre-K-12). 

2) Assessment instruments 

Topic 1 : Native American Tribes 

For lesson 1, the assessment was given to the learners in order to know whether or not they have reached 
the lesson goal/objectives. A quiz contains questions referring to the topic was delivered at the end of the lesson 
(examples of the quizzes and the answers are presented in the students work artifacts). The criterion was the learners 
answered the questions with at least 75% accuracy to be succeeded in this lesson. Before taking the quiz, the 
learners involved in ongoing assessment by doing self-assessment using PBL (project-based learning) checklist tool 
for group K-4. The learners assess themselves by using the PBL check list tool directed by the teacher. 
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Topic 2: Christmas Around the World 

For lesson 2, the rubrics that consisted of statements that reflect learners’ achievements were used to access 
the learners in order to know whether or not they have reached the lesson goal/objectives. The rubric was created 
online by using the RubiStar assessment tool for the writing (Source: 

http://www.makeworksheets.com/samples/nibrics/perwriting.html ) and the Research Process Rubric for the research 
(Source: http://www.uwstout.edu/soe/profdev/mbrics/elemresearchrubric.html ). The rubric for the research was 
done during the research process and the rubric for writing was done when the learners doing their writing tasks 
based on the information from the research. The criteria were the learners did their research and writing tasks with at 
least 75% accuracy to be succeeded in this lesson. The learners involved in ongoing assessment by doing self- 
assessment by using the primary grade self-evaluation teamwork rubric 

( http://www.uwstout.edU/soe/profdev/mbrics.shtml#cooperative ). The learners felt that, as team members, they did 
all their work, helped their group members, listened to their group members’ ideas, shared their own idea, and 
helped their group members to solve the assigned problems. 

Topic 3: Regrouping 

In order to know whether or not the learners have reached the lesson goal/objectives, the assessment was 
given to them. The assessment was a rubric that consists of five statements: computation, approach to the problem, 
explanation, understanding, and organization of thought. The rubric was created online by using the RubiStar 
assessment tool ( http://www.makeworksheets.com/samples/mbrics/math.html ). 

The rubric was scored at the end of the lesson. To success in this lesson, the learners should achieve at least 75% 
accuracy. 

3) Instructional Strategies and Activities 
Topic 1: Native American Tribes 

The teaching-learning strategies that were used for this lesson were expository and project-based learning 
(PBL). Expository strategy is a presentation or telling strategy used during instruction. In the expository learning 
strategy implementation, the teacher informed the learners the names of the Native American tribes and where they 
lived. Then, the learners (individually) listed five of the Native American tribes and allocated on the map where they 
lived. After implementing the expository strategy, the problem-based learning (PBL) strategy was implemented. The 
problem-based learning strategy focused on knowing the issues, considering all possible factor and finding a 
solution. Because all ideas are accepted initially, problem solving allows for finding the best possible solution as 
opposed to the easiest solution or the first solution proposed. Thus, the learners were organized into three groups. 
Each group consisted of three learners. Each group would responsible to search information of one Native American 
Tribes. Then they searched the information on the Internet. They discussed and decided the most appropriate 
information to describe the Native American Tribes that they finding the characteristics, including the food, the 
clothes, and the shelters of the tribes. At the end, they created reports and took the quizzes. 

Topic 2: Christmas Around the World 

For this lesson, telecollaborative learning strategy was implemented. Telecollaborative is a strategy of 
teaching and learning that allows the learning process to be more multi-ways communications, not only between the 
teacher and the learner, but also communication among the learners or among the teachers across the schools even 
countries. The most important part is that the collaborative learning empowering the learners to work together and 
not to compete each other. They learned from one to another. Furthermore, this type of learning supports the 
development of learners ' social abilities. By implementing this strategy, the learners learned from and with others 
via the Internet. They learned how to do research, write a report, and work in groups. The learners were organized 
into groups. The learners researched about Christmas around the world by visiting website related to Christmas 
celebrations in other parts of the world. Furthermore, they communicated online with people in other countries 
(Indonesia and Korea) via Skype. They asked questions and took notes. Finally, the learners made a group report 
and gave it to the teacher. 

Topic 3: Regrouping 

The strategy used with this lesson was problem-based learning. Through the problem-based learning, the 
learners learned how to solve problems regarding regrouping in math, and also they learned how to relate it to their 
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everyday lives. Considering that most of the learners needed intensive and more attentions to master this topic, the 
task was done individually by the learners. The learners listened to teacher’s explanation of regrouping tens and 
ones, counting on tens and ones, and counting by two-digit addition. Then, they practiced one technique by solving 
the problem presented on the PowerPoint application software and on paper. 

4) The Technologies Integration 

For the Native American Tribes topic, the following technologies were integrated in this lesson: computers, 
KidPix application software, and the Internet access to support the teaching and learning processes. For topic 
number 2 Christmas Around the World, the following technologies were integrated: computers with Internet 
connections, KidPix application software and Skype to search for information and to have more practice online. 
Technologies that were integrated in the “Regrouping” lesson were: computer with Internet access to access Math 
online games and the PowerPoint application program. Beside the listed technologies for each topic, hearing aid and 
assistive learning devices (ADL) were provided as needed for the learners to have a better and a convenience ways 
to learn their best. 



Implementation 

In the implementation phase, the graduate students worked with a cooperating second-grade teacher to 
implement lesson plans. The cooperation created a mutual relationship with the teacher, allowing the graduate 
students to apply their instructional design and technology integration skills. The implementation date of the three 
lesson plans can be seen in table 1 below. 



Table 1. Title of the Lesson and the Date Taught. 



Lesson Plan 


Title of the Lesson 


Date Taught 


1 


Native American Tribes 


November 9-10 


2 


Christmas Around the World 


December 1-3 


3 


Regrouping 


November 16-18 



Evaluation 

In the evaluation phase, the student assessment instalments and the learning strategies used were evaluated. 



Findings and Discussion 

The findings from this project focus on the success of the technologies that have been integrated and 
implemented for each lesson. 

Topic 1 : Native American Tribes 

The quiz results indicated that the learners achieved 80%-90% accuracy. This confirmed that the learners 
completed the lesson successfully since he criterion was at least 75% accuracy. The goals were achieved and met the 
content and technology standards. 

The strategies used for this lesson were appropriate. The implementation of the expository learning strategy 
at the beginning of the lesson was very helpful in directing the learners to the next method that was problem-solving 
strategy. Because of this, the learners were more prepared and had description on what they learned and how they 
are going to solve the problems through research. Implementing problem-solving not only developed learners’ 
analytical thinking, but also enhanced their social abilities. The used of computers, KidPix application software, and 
the Internet access, supported the teaching and learning processes. The used of computer applications were aligned 
with both content area and the technology standard for the learners. The teacher integrated the computer application 
into teaching and learning processes appropriately and meaningfully. The learners were able to use those 
technologies to support their studies. With computers and KidPix application software in which the U.S. map is 
displayed on computers screens, the learners were able to locate where the different tribes lived by drawing the 
pictures and labeling where different tribes lived. With computers and the Internet access, the learners were able to 
search information of the Native American tribes, including information about their food, clothing and shelter. This 
technology provides the learners with more resources to get the information related to the topic. The learners were 
organized both individually and in groups. 
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Topic 2: Christmas Around the World 

The criteria were the learners did their research and writing tasks with at least 75% accuracy to be 
succeeded in this lesson. The results show that the learners achieved 80%-100%. On their writing criteria, the 
learners obtained 79%-95%. This meant that the learners completed the lesson successfully. The goals were 
achieved and met with the content and technology standards. 

The strategy used for this lesson was telecollaborative learning. This strategy was used for the first time in 
this class. The teacher and the learners were excited. This strategy was not only enriched students’ knowledge of 
how people around the world celebrate Christmas, but also improved their research by using interview techniques to 
collect data. The learners learned how to write and ask questions, record the data, and write reports. Computers with 
Internet accesses, KidPix application software, and Skype application software were used to support the teaching 
and learning processes. The used of computer applications were aligned with both content area and the technology 
standard for the learners. The use of computers with Internet connections, KidPix application software, and Skype to 
search for information and to have more practice online, supported the teaching and learning processes. The learners 
had more practices and resources to better understand the topic, to improve their writing and research skills and to 
work collaboratively for a better and richer learning content as well. Computers with KidPix application software 
were used to provide the learners with practices writing. The teacher facilitated the learners to use computers with 
Internet access and Skype in searching and adding information related to the topic, and also in working 
collaboratively with others online. The learners were organized both individually and in groups. The learners felt 
that as team members they did all their work, helped their group members, listened to their group members’ ideas, 
shared their own idea, and help their group members to solve the problems. 

Topic 3: Regrouping 

To achieve success in this lesson, the learners were required to achieve at least 75% accuracy. The results 
showed that the learners received 75%-100%. It meant that the learners completed the lesson successfully. The goals 
were achieved and met with the content and technology standards. 

The strategy used in this lesson was problem-solving. The problems were presented by using learners s’ 
name. Moreover, the problems were closely related to their daily lives. By using contextual problems the learners 
easily learned how to solve the problems. The learners were very enthusiastic to learn using the problem-solving 
method and the PowerPoint presentation. Because of this, the implementation of the problem-solving method for the 
Math subject area, particularly in regrouping was appropriate. Computers with Internet accesses and PowerPoint 
application software were used to support the teaching and learning processes. Computer applications used in this 
lesson were in aligned with the content area that is Mathematics. In addition, it was also in alignment with the 
technology standards for learners. The use of computers with PowerPoint application software supported the 
teaching and learning processes. Computers with PowerPoint application software supported learners’ learning 
activities in solving problems and broadening their understanding related to the topic. The use of computers with 
PowerPoint application software supported the teaching and learning processes by providing an opportunity to the 
learners in practicing the regrouping concepts. Through the PowerPoint application software, the problems of 
regrouping were presented in fun and interesting ways. The learners were organized individually. 

Conclusion 

The implementation of the three lesson plans were reflected for future used and development as follows: 
Topic 1: Native American Tribes 

When teaching this lesson again, provide more time for the students to do research in order to solve the 
problems. Also, the selection of the websites used to help the students in searching relevant information need to be 
determined based on the specific tasks. In addition, the integration of computer into classroom needs to be optimized 
and related it to other learning resources. 

Topic 2: Christmas Around the World 

When teaching this lesson again, the time constraint between one country and the others need to be 
considered. For this lesson, the students interviewed the people in the eastern part of the world at 9:00am of the U.S. 
time which means 11:00pm of Indonesian time. In addition, provide more time for the students to formulate their 
questions and how they will organize the questions. 

Topic 3: Regrouping 

Some factors need to be considered when teaching this lesson again in the future. First, adjust the problems 
according to the characteristics of the students, especially their backgrounds. By doing this, the lesson will be more 
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meaningful and effective. Second, provide more time for the students to internalize the concepts individually. Third, 
create or adjust the existing PowerPoint slides that meet with students’ needs. 

Based on the findings and discussion above, the following conclusions were made: 

(1) The lesson plans and associated activities that incorporate methods and strategies of problem solving, 
telecollaborative learning, instructional tools integration and uses of media and online tools for second 
grade students were successfully created. 

(2) The students had experiences to co-teach the three lesson plans designed to enhance the integration of 
technology into second grade classroom. To work with a cooperating second-grade teacher to 
implement lesson plans created a mutual relationship between the graduate students as instructional 
designer and technologist and the teacher. Moreover, this cooperation allowed the graduate students to 
apply their instructional design and technology integration skills. 

(3) The implementation of the three lesson plans that incorporated the learning strategies and the 
technologies in the second grade classroom was successful. 

(4) The reflection of the experiences in implementing the three lessons can be used as references for future 
development and implementation. 

The following are some lessons learned throughout the project that would benefit the graduate students to 
develop innovative lesson plans in the future. 

(1) Systematic phases in developing lesson plans are very important to step-by-step lead the instructional 
designers and technologist and the teachers to develop lesson plans. 

(2) The whole components of lesson plans should be inline. In addition, some factors need to be 
considered when we design and redesign lesson plans, such as the actual time needed to do the tasks 
listed on the lesson plans, contextual learning content in which the lesson plans will be implemented, 
and the integration of the technologies used in the teaching and learning process should be optimized 
and related to other learning resources. 

(3) To be fully integrated, the technologies used in the teaching and learning process should serve the 
strategies that have been selected. 
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Abstract 

As universities continue to increase their online presence, they are challenged to re-evaluate their ways of teaching 
to best utilize the new opportunities that digital technologies offer. While these technologies present a variety of 
methods for online collaboration, virtual worlds offer the added ability to interact within a realistic graphical 
environment, situating learners together in the same space and time. These features give participants a highly social 
and immersive environment within which they can represent their role-play characters through avatars. Integrated as 
part of a university course, the inherent social characteristics of these spaces create opportunities for interactions that 
can increase students’ motivation to leam. Beginning with a discussion of the pedagogical foundations of virtual 
learning environments (VLE’s), this paper describes the instructional situations that these environments are best 
suited to teach. Focusing on four components found to be inherent in such spaces - enabling contexts, resources, 
tools and scaffolds (Hannafin, Hannafin, & Oliver, 1999; Jonassen, 1999) the paper then describes a VLB that 
integrates Moodle and Second Life with SLOODLE to create a rich social constructivist virtual learning 
environment for teaching Security Council role-plays within an International Studies university course. 

Introduction 

What is a Virtual Learning Environment (VLE)? The term can refer to any teaching and learning tool that 
is transmitted online, but to be valuable, the environment needs to include certain characteristics that are conducive 
to learning. Dillenbourg (2000) provides seven useful guidelines for thinking about VLEs during the design 
process: 

• The information space has been designed. 

• Educational interactions occur in the environment, turning spaces into places. 

• The information/social space is explicitly represented. The representation can vary from text to 3D 
immersive worlds. 

• Students are not only active, but also actors. They co-construct the virtual space. 

• Virtual learning environments are not restricted to distance education. They can also enrich classroom 
activities. 

• Virtual learning environments integrate heterogeneous technologies and multiple pedagogical approaches. 

• Most virtual environments overlap with physical environments. 

The 2D virtual environment is currently much more accessible and useful for learning than the 3D 
environment (Thorne, 2009). Learning management systems such as the open source system Moodle allow 
accessibility from anywhere in the world, the uploading and sharing of materials, online discussions and chats, the 
ability to give quizzes, tests, take surveys, record grades, and give and review assignments, and much more. 
However, literature suggests that 3D virtual worlds can be better suited for experiential learning environments 
(Jarmon, Traphagan, Mayrath, &Avani, 2008). 

Three-dimensional (3D) virtual worlds such as Second Life, Croquet and OpenSim have been described as 
desktop interactive virtual reality within a chat environment that "offer the illusion of 3D space, avatars that serve as 
the visual representation of users, and an interactive chat environment for users to communicate with one another 
(Dickey, 2005)". They have been used for a number of educational purposes in various settings and within diverse 
disciplines of study (i.e. media arts, education, health and environment, commerce, language, computing and 
librarianship.) Hew & Cheung’s (2010) review of literature on 3-D virtual worlds in K-12 and higher education 
settings indicates that virtual worlds may be utilized to communicate information between people; to become 
immersed or embodied within a 3-D environment through an avatar; and to act on an object. There are several 
benefits of using this technology. Hew & Cheung's (2010) analysis revealed students enjoyed moving around in the 
3-D space, meeting new people, and having simulated experiences. However, students are averse to inaccessibility 
through older and public computers, issues and expectations of using the chat tool, and unfamiliarity with the virtual 
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world software. Pfeil, Ang, and Zaphiris (2009) studied the usage patterns of teachers and learners in the 3D virtual 
world, Second Life, and identified the following list of challenges that they faced: 

• Signing up students into Second Life (SL): Getting students signed up requires preparation and is time- 
consuming. 

• Orientation and navigation in SL: Navigation and orientation in SL is very challenging for first-time users. 

• Issues of identity: For groups that have already met face-to-face, group dynamics and familiarity have to be 
re-established in SL. Additionally, it is difficult for tutors to identify students in SL, track their progress 
and assess their learning performance. 

• Communication patterns in SL: Switching between different communication channels is problematic, as it 
increases the cognitive load and requires students and tutor to re-adjust. 

• Teamwork: More time without the tutor allows for more socialization, but often, tutors are keen on “being 
here” and observe the student’s learning process. 

• Assessment: Learning in SL often takes place by playing and exploring, but how can playing and exploring 
be assessed? 

• Accessibility: At the moment, SL is totally inaccessible for students with visual impairments. Additionally, 
not only mechanical access, but also the social construction of disability in SL impact on the power- 
distribution within SL. 



Pedagogical Foundations 

Much of the literature on educational uses of VLEs centers on the constructivist paradigm (Dickey, 2005), 
and frequently cites the instructional design theories of learning environments (e.g.,Hannafin, Hannafin, & Oliver, 
1999; Jonassen, 1999). These theories tend to focus on problem solving in ill-defined domains or situations that 
value multiple perspectives over one “correct” solution. The Open Learning Environment (Hannafin, et al.) is based 
on values such as personal inquiry, self-directed learning, and hands-on experiences involving realistic problems 
that have relevance to the student. Similarly, Jonassen’ s Constructivist Learning Environment values learning that is 
based on experiences that are authentic and facilitate conceptual knowledge-making. There is some convergence 
between the methods that these types of theories offer, and they seem to share at least these four components: 

Enabling contexts 

Hannafin, et al. (1999) describe these as externally induced, including methods such as case studies, 
projects, or questions (Jonassen); externally imposed, such as role-plays; or individually generated by students who 
choose relevant questions on their own. Enabling contexts can help activate prior knowledge relevant to the topic at 
hand, help learners choose strategies that they have found useful in the past, or provide background information for 
specific roles that the students are given to play. 

Resources 

Successful learning environments provide a variety of resources that learners can use - websites, videos, 
books, articles, and subject matter experts are a few examples. They can be static (e.g., print resources) or dynamic 
(e.g., interactive charts and diagrams) Learners may have these resources provided to them or may need to research 
and locate their own. 

Tools 



These include information processing tools (seeking, collecting, organizing, integrating, or generating 
tools), manipulation tools, and communication tools (synchronous and/or asynchronous). Information processing 
tools promote many types of behaviors. These include seeking (e.g., researching, using maps and diagrams); 
collecting (e.g., cutting and pasting found information, recording speeches); organizing (e.g., creating outlines); 
integrating (reflecting); and generating (e.g., creating reports or briefs) 

Scaffolds (domain-specific, conceptual, metacognitive, procedural, strategic) 
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Scaffolds may be provided in different ways: by tools, teachers, subject matter experts, or peers. There are 
also different types of scaffolds, including procedureal scaffolds that provide direction, conceptual scaffolds that 
prompt students as they learn content matter, and metacognitive scaffolds. 

Potential outcomes include motivation from learning topics that have real-world implications, deep rather 
than surface learning, and increased critical thinking skills. 

The VLE described in this paper consists of the integration between an LMS (learning management 
system) and a virtual world, each providing some level of support within these four components. The Virtual Model 
UN (VMUN) Learning Environment in particular is built using the open source LMS Moodle and the virtual world 
Second Life. These two programs are integrated using an open source application called SLOODLE (Simulation 
Linked Object Oriented Dynamic Learning Environment), which provides various tools that connect features such as 
course registration, chat, IM, blogging, and assessment between the two environments. 

Role-Play 

Role-plays are effective ways to teach and assess learning (e.g., McCarthy & Anderson, 2000; Ip & Linser, 
2001), with the goal of helping students consider alternative viewpoints, explore opportunities and barriers, and 
develop their critical thinking skills (Belloni, 2008). Therefore, they are well-suited as teaching strategies within the 
framework of social constructivism for virtual learning environments. 

Although the use of role-plays and simulations in the classroom is rising, very little attention has yet been 
paid to the practical application and integration of role-plays into coursework, especially at the university level 
(Asal, 2005). Furthermore, most educational political science role-play simulations are still performed face-to-face in 
the classroom. However, a small number of educators have begun using online discussion forums (either separate 
from or within their organization’s learning management system) or dedicated web applications (e.g., ICONS 
Project, USIP OSP (United States Institute of Peace Open Simulation Platform)) to supplement the more traditional 
approaches. This paper argues that these online role-plays can be further enhanced within the framework of an 
entirely online class through the use of three-dimensional virtual worlds. They offer some important benefits over 
discussion forums, such as ‘presence’ (Bronack, Sanders, Cheney, Riedl, Tashner, & Matzen, 2008) and the ability 
to communicate in real-time chat, IM and voice. According to a recent study involving enactive role-play in Second 
Life, students “valued the embodied experience afforded by the immersive virtual environment (Jamaludin, A., 

Chee, Y. & Ho, C., 2009)." During a recent enactment of a Security Council meeting involving the University of 
Central Florida’s Model UN student organization (Figure 1), one participant noted that she was less nervous during 
the virtual as compared to face-to-face role-plays. This may be a promising area for future research - for example, 
role-playing in a virtual environment such as Second Life may allow more introverted students to communicate 
more and have higher performance outcomes (Gao, Noh, & Koehler, 2008). 
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Figure 1: Security Council Role-Play, January 2009 



The Virtual Model UN (VMUN) Learning Environment 

Model UN can trace its roots back to the Model League of Nations. It is a simulation of the UN General 
Assembly as well as other UN bodies. Students play the roles of ambassadors of member UN states to debate topics 
that are on the UN’s agenda. During the role-play exercise, students learn by researching their roles, making 
speeches, participating in debates, reflecting, and debriefing. These activities require a combination of pedagogical 
components that can include traditional (e.g., lecture, tutorial) and alternative (e.g., discussion, chat) methods of 
instruction. 

VMUN gives learners a realistic environment closely aligned with the United Nations Security council 
building. The learning environment provides enabling contexts, resources, tools and scaffolding to meet the 
learning objectives of the International Organizations course. 
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Two-Dimensional Environment (Moodle) 

Moodle is an open source learning management system (LMS) that is also referred to as a VLE. It is highly 
scalable, making it deployable from small to very large organizations. It includes many features common to LMS’s, 
including activity modules (e.g., forums, chats, wikis, blogs, and databases) to create social constructivist 
environments or modules for more traditional elements such as instructional content and assessments (e.g., choice, 
quizzes, assignments). It has an extensive community of users and developers, extensive documentation, and a wide 
variety of plugins to enhance and add features to the default installation. 

It provides several of the components necessary for an open learning environment. Within the context of 
role-play, it houses descriptions of the scenario, information about the roles to be played, and links to resources 
within and external to Moodle to help the players succeed in their learning objectives. It offers communication tools 
that offer students the ability to collaborate, create alliances, hold secret negotiations, or contact the instructor. One 
of these communication tools is linked to the web intercom Sloodle tool, allowing students within Moodle to 
communicate with others in Second Life. 

VLE Integration (SLOODLE) 

SLOODLE is also an open source project and offers integrative support for both teachers and learners as they 
navigate the digital environment. VLE tools such as identity management to keep track of registered students and 
their avatars, Weblntercom, for real-time, two-way communication between users in Moodle and Second Life, and 
chat logs to archive and retrieve chat in both environments (Kemp, Livinstone, & Bloomfield, 2009) alleviate 
several issues cited in the Pfeil, Ang, and Zaphiris study (2009). 

The first step of the integration procedure is installing the software into Moodle. This currently requires 
back-end access to the Moodle installation to transfer the necessary files to the correct folders and then create the 
database tables by pointing to the Moodle administration page. The exact instructions (which can be found at 
SLOODLE ’s Wiki) are beyond the scope of this paper. 
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Figure 3 The Sloodle Set 



After it is installed successfully in Moodle, the SLOODLE Set can be obtained from SLOODLE Island in 
Second Life. Again, the exact procedure for integrating the set with Moodle can be found on the Wiki. It involves 
rezzing (bringing the object out of your inventory to the ground at your VLE) and then clicking the box and 
following the directions. Figure 3 above shows the box successfully integrated with the Moodle course site. The text 
above the box shows the Moodle website’s address, http://slmodelun.org/moodle, and the name of the course, 
International Organizations. The text “Connect Moodle” and the little button on the top of the Sloodle Set box will 
turn green. 

Once the two environments are communicating with each other, it is possible to click the Sloodle Set box 
and draw out the tools that are included. The VMUN environment uses two of these tools: the Registration tool, and 
the Weblntercom. 

The registration tool (Figure 4) helps instructors identify their students in Second Life by linking their real 
life names to their avatars. This feature allows instructors to keep track of their students within both the Moodle and 
Second Life environments - as noted earlier, one of the frequently cited disadvantages of teaching in Second Life is 
the difficulty of tracking students within the Second Life environment to align them with their assessments and other 
content in the 2D environment. Furthermore, assessments given within Second Life using SLOODLE tools such as 
the quiz chair or Choice are automatically assigned to the correct students in Moodle. 
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Three-Dimensional Environment (Second Life) 

Virtual worlds, Second Life among them, are drawing scores of educators looking for opportunities to 
experiment with experiential and constructivist learning spaces. Further, Dede’s (1995) research implies that 3D 
virtual spaces provide a safe environment in which to learn by doing. However, there are some important 
considerations for design in a three-dimensional space. A recent review of literature on the use of Second Life in K- 
12 and higher education shows that researches found role-playing to be a worthwhile activity as well as a 
convergence of four recommendations among them (Inman, Wright & Hartman, 2010): 

• Establish a clear connection between the course objectives and activities in Second Life. 

• Incorporate technical training and support into any planned Second Life activity. 

• Create scaffolded learning activities for students so they can practice inside Second Life and acclimate to 
the virtual environment. 

• Design and construct different spaces to encourage different types of student interaction. 

The primary tools and scaffolding required for role-play are related to communication and social 
connections between the learners and instructors. Moodle provides multiple communication tools: chat, email, 
discussion forums, wikis and blogs among them. Depending on the learning objectives and the preferences of 
the instructor, each of the tools can provide an important function of a role-play simulation. The main role-play 
area of the VMUN environment is the Security Council Room. In Figure 5, the Security Council Room is 
shown with a Weblntercom (turned on and off by a click of the mouse) set up to record all chats during the 
meetings. Separate chat sessions can also be set up within Second Life to provide space for private meetings 
between groups. This is designed by creating chat spaces that are beyond the parameters of each other (to 
ensure that other groups cannot listen in) and Weblntercom boxes can then be set to record only for the group. 
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Figure 5: Web Intercom logs chat and connects with chat module in Moodle 



Manipulation tools are also important during role-play. For example, Security Council members require 
some method of calling attention to themselves so that they may get the floor and speak. Tools such as buzzers that 
can be clicked or banners that can be raised can be used for this purpose. 

These are some of the many tools and scaffolds that are available for the design of a virtual learning 
environment. 

Conclusion 

VMUN Learning Environment is a work in progress. It is based on a social constructivist pedagogical 
foundation and designed to include four components that are commonly found in virtual learning environments 
(Flannafin, et ah, 1999). During the VMUN environment’s design process, role-play simulations will be run for 
international studies and political science courses. It is the intent of the authors to begin the next step of the project - 
evaluation - in the Spring semester, with a planned role-play simulation for an American University Human Rights 
course. To ensure that the VMUN’s design is correctly aligned to its learning objectives, the evaluation process will 
be based on Hannafin, Hannafin, & Land’s (1997) conception of grounded design, with concerted effort put into 
complying with its four conditions: 

• The design must be based in a defensible theoretical framework: the social constructivist framework is a 
proven and defensible framework. 

• Methods must be consistent with the outcomes of research conducted to test, validate, or extend the 
theories upon which they are based. 

• Generalizable - although the Virtual Model UN environment is a specific real-life environment, the 
methods themselves are generalizable to other teaching strategies within the social constructivist 
framework, using the components inherent in these environments. 
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• The design must be validated iteratively through successive implementation - this is the next step in the 
VMUN project. Each implementation of the environment will provide opportunities to “continuously 
inform test, validate, or contradict the theoretical framework and assumptions upon which” the 
environment is based (Hannafin, Hannafin, & Land, 1997, p. 103). 

Aligning design to these concepts does not guarantee success. The concepts themselves may be found to 
need refinement as VMUN is iteratively validated. As Hannafin, Hannafin, Land suggest, it is important to practice 
the social constructivist ethic within the design process itself by avoiding restricted or rigid perspectives about the 
methods employed and evince a willingness to adapt and evolve every part of the development cycle. 
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introduction 

Enrollment in online classes is increasing every year as more and more people are returning to college due 
to the current economic downturn (Parry, 2010). Colleges are offering online classes in several different delivery 
formats, including the use of online 3D multi-user virtual environments (MUVE). Research has shown that the 
concept of identity plays a big part in regards to learning. For example, students who are in MUVEs often feel that 
whatever happens to their avatar in a MUVE happens to them in real life too. This sense of identity leads to 
immersion within an online environment, which many educators believe causes students to learn better (Gaimster, 
2008). 

Atkinson (2009) contends that a connection exists between students’ real life identities and their online 
ones. What is most interesting is how identity affects their actions and learning in an online environment. The 
purpose of this research is to understand how students choose to identify themselves within a 3D virtual 
environment and how their virtual identities impact the way they behave and learn in this environment. 

Literature Review 

In real life, identity encompasses a number of factors in regards to how people see themselves, and students 
and educators are no exception. These factors include “social, cultural, and biological characteristics, as well as 
shared values, personal histories, and interests” (Buckingham, 2008). Within Second Life (SL), a popular type of 
online MUVE, the ability to easily change appearance as well join and create social affinity groups adds an 
additional layer to the concept of identity within the virtual environment. Residents of SL are provided the 
opportunity to assign themselves a name as well as different characteristics to their avatar, such as gender, race, and 
social class. The only difference is instead of text-based descriptions, individuals are now provided with 3D 
renderings of their identity as well as spaces to post their profiles to provide further depth to their online identity. 
Furthermore, they are provided the ability to construct 3D interactive objects in their environment to create their 
own unique living situation (Foster, 2007; Sanchez, 2009). 

All of these factors reveal the need for online educational researchers to study students in 3D virtual 
environments. Some studies have already looked at identity and learning in 3D virtual environments, such as the 
study from Alessandra Talamo and Beatrice Ligorio (2001) which looked at children using 3D virtual environments 
and how their identity and roles changed while participating in a particular learning project. Sherry Turkle has 
written many article investigating identity in virtual environments (Au, 2008). Flowever, little research has 
specifically examined identities of those in the educational community who use 3D virtual environments for 
learning. 



Methodology 

The University of Flawai’i currently offers several types of classes online for students to choose from. 
Sixteen of these classes are taught within SL. These classes include Psychology courses, Library and Information 
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Sciences courses, and Second Language Studies classes. One of the University of Hawai’i classes taught in SL is an 
Educational Technology elective where students learn about building learning objects within a 3D virtual 
environment. The researchers of this study surveyed students of this course taught in SL during summer 2009. 

The survey questions were based on a survey from Diehl & Prins' (2008) study about identity in SL. In that 
study, the authors studied SL’s potential for “the construction of identity and development of intercultural literacy.” 
The authors studied the cultural identities of the participants, types of places they visited, the languages they use, 
and their views toward intercultural understanding. In this study, the authors concluded that “participation in SL 
enhanced [the] participants’ intercultural literacy” and that they “used their avatar’s appearance to construct shifting 
cultural identities.” 

In comparison, the researchers in this study looked at how the participants’ presence and identity within the 
SL environment affected their learning. However, instead of using convenience sampling when choosing 
participants as the Diehl and Prins study did, the researchers in this study deliberately chose the students who were 
in the Educational Technology class to participate in the study. Additionally, the researchers examined the 
participants’ identity in regards to their characteristics as adult learners. This study focused more on personal 
characteristics such as aging and life phases, situational characteristics, and the types of learning involved rather 
than on the participants’ culture. Furthermore, the researchers chose to only use surveys instead of the three 
methods that Diehl and Prins used to gather data, which were surveys, interviews, and observations. 

The three research questions for this study are: 

1 . How the participants in the class saw themselves in regards to identity? 

2. What affordances they had in the SL environment and how their identity affects these affordances? 

3. How their identity impacts their learning within this environment? 

The participants first answered basic demographic multiple-choice questions about their sex and marital 
status in the first part of the survey. Of the nine participants who took the survey, six of the nine were female while 
the other three identifies themselves as male. The participants’ ages ranged from mid 20s to mid 50s. Six of the nine 
also stated they were married, while two participants labeled themselves as single, and one was divorced. As for 
their education level, seven out of the 10 identified themselves as graduate students studying in the field of 
Education, while the other two only labeled themselves as “professionals” in the field of education. In general, all 
the participants were within the field of education in general. 

The second part of the survey focused on open-ended questions about the participants’ identity and 
activities they participated within SL, including in-world activities that were not class related. This part of the survey 
allowed the participants to answer in paragraph form where they could disclose as much or as little information as 
they wanted. 



Analysis and Findings 

Results of this survey were quite informative. The researchers analyzed the data based on the students’ 
identity, their affordances, how their identity affects their actions, and how their identity affects their learning. A 
discussion of the results follows. 

Identity 



When asking the participants about their identity, the researchers discovered interesting results. The 
students chose to talk mostly about their identity in regards to appearance, labels, and community affiliation. It 
seems that these characteristics are most important to the participants. 

With regards to identity itself most of the participants stated that SL has no influence over their real life 
identity. Instead, their SL avatar is more of an extension of their real life identity. As one participant said: 

“There is research that shows in online forums that some learners interact, communicate, and behave very 
differently than they do face-to-face. If that is true for me, I am not aware of it. It feels that I’ve just 
extended my own personal identity into a new arena.” 

In other words, the participants felt that their SL and real life identities are the same and that no disconnect existed 
between the two. 

When asked about their SL identity in particular, the participants in the study focused mainly on their 
appearance, which emphasizes the need for the researchers to ask specifically about the other aspects of their real 
life identity in future studies, such as their occupation and personal history. Of the data the researchers did collect, 
they discovered that the participants are not particularly concerned with appearances. Even when they changed 
appearance on occasion in SL, they stated that it did not affect the way they feel. 
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Moreover, race did not seem as important to the participants in regards to avatar creation. Although the 
participants in the study attempted to make their avatar as close to their real life race as possible, it seems that the 
factor of race was not a major factor in their use of character creation. As one participant mentioned: 

“I have no idea what race my avatar is or how closely that resembles my real coloring. After all, in real life 
people are just shades not blocks of easily distinguishable color.” 

Most of the participants chose not to experiment with their avatar’s race either. They also felt the SL avatars could 
not fully represent race perfectly. Interestingly, some of the participants added that they did not feel comfortable 
taking on another identity that was not similar to their own. 

Generally speaking, appearance was not a motivating factor in regards to learning. However, the 
participants did enjoy the ability to “see” one another in the class or at least a physical-looking digital representation 
of one another verses having them represented through only a text-based name as with most distance education 
courses. One participant provided an interesting comment: 

“The biggest surprise for me was a sense of presence. 1 didn’t expect to start of feel as fully emerged in the 
environment and as I ended up feeling within the first.” 

In other words, the immersion aspect of learning in SL was more important rather than the aesthetic aspect of it. 

In addition to their physical traits, the participants did reveal other details about their identity. Some of the 
students identified themselves with labels including “parent,” “Goth,” and “constructivist.” Most of the students, 
however, who took the survey, defined themselves as educators throughout the survey. It seems that their profession 
as an educator is their identity in a sense or at least a major part of it. 

Affordances in 3D MUVEs 

When asked about the affordances that they have in SL in general, most of the participants in the study 
focused on the affordances associated with education while defining themselves as students or educators. Their 
responses focused on participating in activities, joining groups, following acceptable practices and guidelines within 
the environment, visiting places, and navigating the SL environment. 

Throughout the study, all the students stated that they participated in SL for mostly academic or class 
related purposes the majority of the time while logged in. For the most part, the participants felt that MUVEs helped 
educators professionally and they also felt that SL experiences were transferable to real life. As one participant said: 
“From this point out, I see my presence in Second Life, much the same as I see my presence in Twitter, 
conferences, online social forums. It is becoming part of my personal learning network that I can use to 
both personally improve my professional knowledge, as well as share with other educators as an emerging 
place for them to consider for their own personal use, and as a class resource.” 

The participants in the study did mention that they participated in some non-education related activities though. 

They stated that some of these activities included socializing and visiting calm, stress-free places. 

Students also tended to form relationships and communicate mainly within the educational community and 
on occasion ventured outside of that community. Some joined groups in SL on their own without prompting from 
the instructor, but it seems that those who were members of SL before the class started tended to engage in this 
practice more than the students who signed up for SL only for the purpose of attending the class itself. 

Those who had to join SL, in order to attend their class during the summer, usually joined the groups that 
were required of them. If they did join groups that were not required of them, they mainly join educational groups. 
One of the participants reflected that: 

“I join some groups that were related to educational technology and learning. [The International Society for 
Technology in Education group] is a good example of this. I never really considered joining groups that 
would fall outside of a professional focus.” 

The students tended to participate mainly within their own education-themed events as well. 

The participants saw SL as an extension of their personal learning network also. In other words, educators 
feel that they are part of a community in SL. They also believe that as a community they need to abide by the rules. 
Ergo, they saw themselves as members of a “community” in SL. Interestingly; the participants felt that rules in 
general were necessary within the community groups they were in also. 

When the students were asked about visiting other communities in SL, they talked about educational sites 
or places where they can buy virtual items for free. The UH island, “freebie” sites, and educational sites are a few of 
the places they talked about. The students did not mention that they looked for ways to make money in-world. This 
is probably why they went to places that offered virtual items for free. The researchers believe that perhaps this is 
also a reflection of educators in real life who are not necessarily known for their wealthy lifestyle. 
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On occasion, the participants visited places in SL for interaction and relaxation. However, they mostly 
visited educational places required for class. As for technical issues, some found navigation around the SL 
environment difficult: 

“When other tools are available in real life and would be much quicker to use, I doubt I would use second 
life instead. I think perhaps in 10 years or so perhaps when there are more resources available that are 
easily searchable, customizable, and immediately useable, would I consider using it in the classroom. The 
reason is because SL is not a tool that I can quickly implement and there is a steep learning curve.” 
Therefore, they felt SL’s search tool was not easy to use and had a hard time finding and navigating places they 
specifically wanted to visit. 

As a whole, the participants experienced a sense of community and presence, same as non-educators in SL. 
They were more adept to staying within their own affinity groups. In addition, their actions in-world was practically 
similar to the type of behaviors educators’ exhibit in real life such as following the rules and looking for free 
materials to help them with their educational goals. 

Identity and Learning 

By and large, the participants in the study saw themselves as part of a learning network. They believe SL 
helps them learn together with others and feel they are part of a global community. One participant remarked that: 
“Being in a place like SL let’s you really appreciate how ‘small’ our world is. SL, like the Internet, truly is 
the ‘Global Village.’” 

In other words, SL enhances a sense of self within a social community. 

Moreover, most of the participants believe SL is a useful educational tool, yet they are concerned about 
open access to everyone. The participants felt the need for accessible, collaborative, and community learning for 
everyone. As one participant mentioned: 

“Yes, I see a great level of utility for experiential learning and for collaborative in SL. I think that the level 
of tech skills and the computer and bandwidth issues limit its scope for now.” 

This may have to do with the culture of academia, which emphasizes equal access and education to everyone, in 
contrast to the private sector, which emphasizes success by those who are financially ahead. Furthermore, they see 
SL as a place for resources that brings another dimension to learning. They enjoyed the fact that students could try 
out simulations for career purposes. 

SL also allowed the participants to comfortably ask questions since it provides users with various ways to 
communicate. These include local chat, private chat, voice, and private calls. As one participant said: 

“Until the class we took this summer, I did almost no communication at all [in Second Life]. What I found 
now is that I feel much more comfortable asking questions of people I don’t know, and fell more 
comfortable setting up small sessions with either voice or chat.” 

Another interesting aspect is that the participants mostly chose to use the chat feature the most instead of voice when 
communicating. Probably the use of chat allows the participants to feel more comfortable communicating among 
each other since it allows users to think about what they want to communicate before typing it out, whereas voice 
puts them on the spot. 

Along with have the ability to communicate with one another in-world, the participants appreciated the 
time aspect of SL: 

“I think that SL is useful for informal education as the array of resources is so broad. While sometimes 
finding something is difficult, there are a great many sites to visit that offer self-paced informal education.” 
In other words, they felt that they could explore and learn at their own pace with or without assistance. Since time is 
often a valuable asset to busy educators, their desire for self-paced learning is understandable. 

Conclusion and Implications 

Identity and affordances seemed to play an important factor in the overall learning of the participants. The 
main characteristic of identity that seemed to affect their learning most was their identity as educators in real life in 
comparison to their race, gender, or age. Ways that their identity affected their learning include who they chose to 
communicate and associate themselves with, their pace of learning, and where they visited on their own. 

As stated earlier, the majority of the students in the class surveyed mainly stayed within their own 
community and rarely ventured “outside” their own group. This brings up the questions, “Does this prevent them 
from learning from others outside their group?” or “Does remaining within their group provide a safe haven for 
learning where rules and boundaries are more adequately controlled and monitored, allowing for safe learning?” 
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Implications from the survey reveal that the need for learning objects that promote communication, 
collaboration, and “sense of place” or immersion is a must when designing lessons, course materials, and 
environments in MUVEs for students. Also, instructors need to take into consideration the identity of the students 
they are working with when designing course instruction. Although survey results from the class revealed much in 
regards to how educators’ identity affects their learning, more research is needed to see if this a common trend 
among other types of educators as well. 

The researchers believe they need to ask more about the participants’ occupation next time as well as other 
aspects of their real life such as their life phases and developmental stages of the participants as well as the type of 
learning they are engaging in within MUVEs. This will help the researchers focus their study on particular aspects 
of identity that may affect adult learners. In addition, the researchers believe that applying the Characteristics of 
Adidt Learning model from K.P. Cross will help ground them more in their study of identity in this group in future 
studies. 

Furthermore, the researchers believe that possibly SL may appeal more to certain types of educators than 
others. In other words, SL may not appeal to all educators who try it. More research is needed to determine which 
types of educators enjoy using MUVEs for learning and socializing. Perhaps future research should examine 
educators’ personal characteristics such as the subject area, and grade level they teach as well. 
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Abstract 

The purpose of this study was to improve the quality of guidance and counseling management through Networking Model. 
The model was developed based on students competence and it strictly required collaboration among teachers, counselor, 
and parents in helping students to solve their problems. The methods used was research and development model of Borg 
and Gall. The procedures conducted were: (i) premilinary surveying to produce software to be used in assessing students’ 
need; (ii) designing networking model chart; (iii) implementing and evaluating the implementation of the model in several 
high schools. Data were collected by identifying and analyzing how the networking model improved the quality of 
guidance and counceling management. The results indicated that the networking model can improve the quality of the 
management of guidance and counceling in the way that the services provided were based on data and it encouraged 
collaboration among different expertises in helping students to solve their problems. 

Introduction: 

The reposition of guidance and counseling which has been declared by the Minsitry of Education in Indonesia, from 
merely a support to learning program to be an integral part of teaching and learning process has resulted in the need to 
replace the old model with the new one. To be able to accomodate its new function, the new model should have 
characteristics such as the abilities to: a) manage all activities integratedly, b) manage information well, c) solve students 
problem efficiently, and d) organize some connected persons and activities simultaneously. Preliminary studies found 
that networking model has been proved to be theoretically meet the requirements. However, the model has not been tested 
necessarily to prove its practical efficiency. In general, the purpose of this study was to improve the quality of guidance and 
counceling management in some schools. Specifically, the objectives of this study were : (i) to design a software by which 
students need and competence will be assesed and analysed (ii) Based on the data obtained in the software, to design a 
model by which all relevant professional persons (expertises) are collaborating in a network system to help students in 
solving their problems. 

Literature Review: 

Guidance and counceling in high school is school services that apply students understand themselves and environment 
and act effectively to be live in a happy life. The services area such as : how the students develop to believe in God, to be 
mature as a woman and man, to be healthy, to learn in university, have a certain role in community, emotionality, socially 
and intelectually muture. The purpose of guidance and counceling program are designing learning situation so that every 
student develop their competencies. The Guidance and counceling program divide as : service, support, assesment and 
controling. The management system is shown in figure 1. 

The concept used to design the new model in this study was taken from the theory of educational management and the 
theory of guidance and counseling technology at school [2] . It is stated that the success of guidance and counseling 
management is indicated by criteria such as: (i) planning which is based on students need map, (ii) efficient organization of 
people and activities, and (iii) motivating counseling and countinuous control 1 " The map of students need comprise 
aspects such as students’: (i) self competence, (ii) social competence, (iii)learning competence, (iv) career competence, (v) 
family competence, and (vi) religious competence^ 1 . Generally, there are five aspects to be considered as related to 
networking model in counseling. They are: (i) level of service which comprises: preventive session, pre counseling session 
and teraphy session; (ii) direction of service which includes: actualization existence, prevention, rehabilitation, 
development, solving and advocation; (iii) service mode which covers: orientation, information, placement, learning, group 
guidance, group counseling. individual counseling, consultation, and mediation; (iv) form service such as: class room, 
group, and individual; and (v) expert, such as: counselor, teacher, parent and other profesional. The New System is shown 
in Figure 2. 

Methodology: 

The method used in this study was research and development which was adopted from Borg, Walter (1983). The 
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procedures were as follows: (i) preliminary survey, (ii) model development, (iii) model testing, (iv) model validation, and 
(v) model socialization. This method was done in three stages. A preliminary survey conducted was data collecting on 
students need and competence. Data collection was performed through the need assesment software and group discussion. 
The data obtained were descriptively analysed to be further used in mapping students’ profile. The networking model based 
on students competence was resulted from the preliminary survey. The research question was: how does the networking 
model improve the quality of guidance and counseling management? The model tested is shown in the Fig. 2 and the stages 
of research and development are shown in Fig. 3. 



Student Developmental Task 




Figure 1 : Management Guidance and Counceling in High School (Depdiknas,2002) [1] . 
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Figure 2 : Guidance and Counseling management model 




Figure 3 : Networking Management Model 
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Figure 4 : The Stages of Research and Development 



Population in this study was high school in Medan City, Indonesia in year 2009. Samples was collected purposively 
with characteristics such as follows: the school has one counselor for 150 students, applicable to cooperative in 
networking model, new school and the school in urban society. The state high school IX of Medan was found to be 
relevant to those characteristics. The samples of students were chosen randomly for first grade with amount 35 students. 
The data were collected through data base, need assesment’s instrument and counseling report. Methods applied in this 
study were interview and focus group discussion between teachers, school principles, and student’s parents. Collected 
data were analyzed using percentage method, in addition, student’s need asesement were analyzed using software 
application. 



Results and Discussion: 

From this study, it has been found that the result of software application could be used as a foundation to networking 
model design. The model was used in mechanism of counseling management as instructed. The different procedures of 
conventional models from Networking model is shown in Figure 5. 
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Figure 5: The Management Model Procedures 
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Results showed that the procedures of new networking model were found to be simpler and shorter than convetional 
model. In terms of time duration, the new model was only need 30 minutes to get student database, however, the 
conventional model was need 3 days for each student. In terms of database input, the new networking model was found to 
be easier than the conventional one. The advance of networking model in terms of out put results, it was pop up with table 
and graph, whereas could not supported in conventional model. Some of the data base performance were shown in Figure 
6.1 to 6.4 as follows : 



Ee 



=============================== LAPORAH IHDIVIDU ======== 

DATABASE: C:\Program FilesNBCINDataSis ( SLTA) \Data\ SMA9 



Hama Lengkap: BUDI SURYA LUMBAH RAJA 
Ho. Induk: 1 
Kelas: 1.1 



2. Hama panggilan: BUDI 

3. Jenis kelamin: L 

4. Tempat lahir : MEDAH 

5. Tanggal lah.it : 29/03/1991 

6. Agama: 1. Islam 

7. Kewar game gar aan: IHDOHESIA 

8. Suku bangsa: BATAK 

9. Diterima tanggal: 18/07/2005 

10. Pindahan dati: - 

11. Anak ke 2 dari 3 bersaudara 

12. Bahasa sehari-hari: 1. Indonesia 

13. Alamat tumah: JL.PERIUK HO 66 MEDAH 
Kode pos: 

Telepon: 4149728 

14. Alamat libur: 

Kode pos: 

Telepon: 

15. Siswa tinggal dengan: 5. Ibu k an dung 

16. Jarak dari tumah ke sekolah: 3. 6-10 km 

17. Berangkat ke sekolah: 2. Pakai kendaraan umum 

18. Kendaraan ptibadi yang dimiliki: 1. Mobil 

19. Kondisi penerangan tumah: 1. Baik 

20. Kondisi tempat tinggal: 1. Permanen 

21. Ruang tidut: 2. Bet sama-sama 

22. Lingkungan tempat tinggal: 6. Lain-lain 

23. Kondisi ruang belajar: 4. Lain-lain 

24. Perlengkapan belajar : 1. Lengkap 

25. Hama ayah: S . LUMBAN RAJA (ALM) 

26. Tempat lahir ayah: SIAHTAR 
Tanggal lahir ayah: 14/04/1960 

27. Agama ayah: 2. Kristen Protestan 

28. Pekerjaan ayah: 



Ijft Simp an | ^ Celak | | ^ OK | 



Figure 6.1 : Students’ identity input 




Figure 6.2 : Students’ Emotional 
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Figure 6.3 : Graph of Student’s Individual Problem 




Figure 6.4 : The Distribution of Student’s Individual Development 
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Socialization networking model were applied to to school’s principles, teacher , counselor and students. Results showed 
that the application of software was easy to operate and user friendly. All of them could use the software. Networking 
model could helped students to know what their counseling needs and they could go to the counselor for counseling 
activity. 

At last, the result showed that networking model could build an effective and efficient management of guidance and 
counseling. In addition, the model could successfully improve the guidance and counseling activities to help students in 
solving their problem. The data obtained from the software, i.e students need assesment that could be used to design 
networking chart. The chart was useful in designing service mechanism. The counselor who evaluated this model found 
the model could be used to solve students problem effectively in a case of conference system. The teachers commented 
that this model gave them a chance to understand their students more and as the result, they were able to integrate learning 
activities and guidance counseling process. 

In addition, the principal found that the model could be used to not only in guidance and counseling activities program, but 
also to the whole school programs. Furthermore, this model could give a chance to teachers, counselors, parents, and 
experts in communicating with each other, especially in student case conference. Based on the results, it could be stated 
that the networking model could overcome the weakness of the traditional approach in guidance and counseling in the way 
that: (i) the services provided in the model were based on comprehensive assessment of students’ need and competence, (ii) 
the activities were more controlable since any single decision was made based on data, (iii) the cooperative interactions 
between counselor, teacher, parents were developed, (iv) the sense of responsibility as a professional counselor was grew 
among the counselors, and (v) the more pleasant and conducive atmosphere was established. 

Besides that, the networking model provided some beneficial contribution to students, which was never achieved in the 
traditional model, in the way that it: (i) increasing students’ understanding of guidance and counseling activities, (ii) 
promoting interaction among peers, counselor and lecturers, and (iii) developing self confidence to be successful in careers, 
and (iv) helping the students in solving their problems efficiently and effectively. 

Conclusions: 

The networking model could became an alternative response towards the government decision in repositioning the 
guidance and counseling activities from merely a support to learning program to be an integral part of teaching and learning 
processes. The new model at the same time was found to be strengthened the role of guidance and counseling in Indonesian 
schools. 
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Abstract 

A new discipline, mLearning (i.e. mobile learning), has formed by pairing what was once considered a 
disruptive technology, i.e. cell phones, with learning. There have been attempts from researchers to establish an 
mLearning Instructional Design Theory (Sharpies et al., 2009), but the field is far from agreeing on one mLearning 
Instructional Design Theory. This paper provides an overview of mLearning and instructional design theories, and 
presents a framework for mLearning instructional design theories that instructional designers can use to design and 
develop instruction on mobile devices. 



Introduction 

The use of mobile devices continues to evolve and enter many facets of daily life. The field of education 
and learning is no exception; many educators are eager to explore the potential of the devices that allow learning in 
the mobile age. The instructional designer has an essential role in mLearning because it is the instructional designer 
who will design the learning content. With little to no viable mLearning instructional design theories established, it 
is difficult to measure the effectiveness of instructional design on mobile devices and to measure the learning taking 
place. Shuler (2009) states that “only once we have a theory for learning with mobile technologies in different 
settings and different populations can these devices be used to their frill potential'’ (p. 26). This paper helps to build 
on mLearning research by analyzing the current instructional design theories for mLearning and providing a 
framework with which to evaluate and build an instructional design theory for mLearning. 

Overview of mLearning 

mLearning can be described as the “intersection of mobile computing and eLearning: accessible resources 
wherever you are, strong search capabilities, rich interaction, powerful support for effective learning, and 
performance-based assessment. It is eLearning independent of location in time or space” (Quinn, 2000). However, 
mLearning as eLearning is only one perspective of mLearning. The other three, technocentric (i.e. focus on mobile 
devices), augmenting formal education, (i.e. mobile learning as a tool in formal education), and learner-centered (i.e. 
focus on mobility of learners), offer a different perspective on the purpose of mLearning (Winters, 2006). These 
perspectives can be compared to the four categories of effective learning: learner-centered, knowledge-centered, 
assessment-centered, and community-centered (National Research Council, 1999). Regardless of whether you agree 
with one or all of the perspectives of mLearning, one thing is clear: mLearning can take many different shapes and 
forms. 



Current theories of mLearning 

There are three existing theories and/or frameworks of mLearning that are described in this paper. Each 
adds an important perspective on the instructional design process. Sharpies (2000) developed a theory of lifelong 
learning and a design process, Shih & Mills (2007) developed a mobile learning model, and Sharpies et al. (2005) 
and Wishart (2007) developed evaluation frameworks for an mLearning theory. 

Sharpies theory of lifelong learning involves eight features. Any instructional design for lifelong learning 

must be: 

• portable: available whenever. 

• individual: adapting to learner’s abilities. 

• unobtrusive: retrieve knowledge without obtruding the situation. 

• available anywhere: enabling communication. 

• adaptable: to learner’s evolving skills and knowledge. 
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• persistent: possible to manage learning through a lifetime. 

• useful: suited to everyday needs. 

• intuitive: easy to use by novices. 

(Sharpies, 2000) 

mLearning must also follow a model of motivational design. One of the most appropriate models is 
Keller’s ARCS model: attention, relevance, confidence and satisfaction (Shih & Mills, 2007). Shih & Mills’ mobile 
learning model includes Keller’s ARCS model. Shih & Mills’ model involves the following five steps: 

1. attention: sending a multimedia message to mobile phones to trigger and motivate learners. 

2. relevance: searching the Web for related information using embedded hyperlinks. 

3. relevance/confidence: discussing with peers through text, voice, picture, or video. 

4. confidence: producing a digital story of what they learn by audio or video diary. 

5. and satisfaction: applying what they learn in a simulated environment. 

(Shih & Mills, 2007) 

Sharpies et al. (2005) and Wishart (2007) both developed evaluation frameworks for an 
mLearning theory that involve the following questions: Is it (i.e. mLearning theory) significantly different from 
current theories of classroom workplace or lifelong learning? Does it account for the mobility of learners? Does it 
cover both formal and informal learning? Does it theorize learning as a constructive and social process? Does it 
analyze learning as a personal and situated activity mediated by technology? Does it analyze the dynamic context of 
learning? These questions provide a lens through which any mLearning theory can be critiqued and analyzed. 

Limitations with the Current Theories of mLearning 

It is important to acknowledge that the above three theories and/or frameworks have limitations that must 
be addressed before a general framework of mLearning can be established. First, Sharpies’ design process provides 
an in-depth look at the instructional design process for mobile devices, but it is unclear if this process is for formal, 
informal, or lifelong learning. Instructional designers must determine the differences in these types of learning 
situations in order to accommodate our design process. 

In the Shih & Mills model, motivation provides a foundation for the design. Although it is imperative that 
we involve motivation to gain attention, relevance, confidence and satisfaction, instructional designers must not 
forget the layers of design (Gibbons & Rogers, 2009). It is just as important that we design for the content and data 
management layers. Just-in-time and customization features are needed to support different learning styles and 
abilities. 

Finally, Sharpies et al. and Wishart’s evaluation frameworks do not take into account the four perspectives 
of mLearning discussed earlier. It is necessary to think about the perspective when designing for mLearning. 

A Framework for Instructional Design Theories on mLearning 

A framework for instructional design theories on mLearning must consider the four perspectives on 
mLearning. It must include aspects of theories on lifelong learning, motivation, and evaluation. It has to fit within 
instructional design theory and the layers of design. The framework provided in Figure 1 includes these 
considerations and can be applied to any instructional design theory on mLearning. 

Even though the arrows in the proposed framework indicate a process, this is by no means systematic. The 
arrows are only used to suggest a general course of action for a novice instructional designer. The novice 
instructional designer might start with the learning situation. Depending on if the learning is formal, informal, or 
lifelong, the instructional design theory could be very different. From there, the designer looks at the perspective. 
Within formal learning, for example, the designer could have a very device-centered, or learner-centered perspective. 
Again, these differences would change the instructional design theory. Theories of learning are based upon the 
learning situation. However, in general, mobile learning will follow the theories of lifelong learning. Next, 
motivation must be considered. What is the designer doing to gain the learner’s attention? Is the designer using push 
features (i.e. text messages)? Once initial planning has been considered, the designer can start a design process. This 
could vary from user-centered design to a very traditional ADDIE process. Evaluation will help the designer 
determine if goals have been reached. 
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t 
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Tv 
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confidence, satisfaction) 



Theories of 
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(portable, individual, 
unobtrusive, available, 
adaptable, persistent, 
useful, intuitive) 



Figure 1: A Framework for Instructional Design Theories on mLearning. The arrows represent the path a novice 
instructional designer might follow. 



Conclusion 



mLearning is quickly becoming a bridge between formal, informal and lifelong learning. The instructional 
designer has an essential role in mLearning because it is the instructional designer who will design the learning 
content. After investigating the current instructional design theories and issues, this paper provided a framework 
through which instructional designers can analyze and use instructional design theories for mLearning. Although 
this framework must be tested and studied further, it provides a starting point for instructional designers in the field 
of mLearning. Instructional designers can use this framework to create and design meaningful instruction. Future 
studies must be completed to explain and test this framework. 
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Abstract 

A collaborative image-based bookmarking tool enables multiple educators to work together to create, save, 
and share a collection of resources for use in a learning activity. The collection of resources can be accessed in any 
classroom and provide a direct link to many different Internet resources. This paper presents the Cube, an example 
of such an image-based bookmarking tool, the design process, and questions that must be researched to expand the 
body of knowledge in this area. 



Introduction 

Social bookmarking has been a hot Web 2.0 topic ever since the creation of delicious (2010), a 
bookmarking tool that enables users to tag, save, manage, and share text-based representations of web pages through 
an online community. In K-12 education, teachers use bookmarking tools to store lesson resources and artifacts that 
they can access during a lesson (Wepner & Tao, 2002). However, there are problems with using traditional 
bookmarking tools (e.g. Internet Explorer, Firefox, Safari) or social bookmarking tools (e.g. delicious) in the 
elementary classroom. First, teachers have resources stored in many different places and need a place to aggregate 
their Internet resources in one place. Second, teachers lose the attention of their students while they search for 
bookmarked resources. Finally, traditional text-based bookmarking systems do not enable teachers to preview a 
resource before accessing it. This paper introduces an image-based bookmarking tool that provides teachers with 
solutions to these issues. 



Rationale 

In About Face 3: The Essentials of Interaction Design, Cooper, Reimann & Cronin (2007) explain that 
toolbars in many popular software programs (e.g. Microsoft Office Suite) use butcons (i.e. icons that also serve as 
buttons) because recognizing images is faster than reading text. An image-based bookmarking system enables 
teachers to organize Internet resources with clickable visual previews of the resources. By using clickable visual 
previews, teachers can speed up the recognition process, thus, saving time and keeping students on task. Using 
clickable visual previews may also provide a novelty effect for students (Gall et ah, 2003). In other words, the 
‘newness’ of the visual representation of bookmarks may keep students on task longer than the traditional text-based 
representations. 



Goals of Design 

Keeping the rationale in mind, the Cube was designed using a goal-directed design process (Cooper et ah, 
2007). The goals for the first prototype were as follows: 

• Use images as bookmarks to enable teachers to speed up the recognition process and keep students 
on task, 

• Increase collaboration between and among teachers on lesson resources by providing the ability to 
edit bookmarks on one cube at the same time from different locations and computers, 

• Enable teachers to save a set of resources in one location (i.e. one cube), 

• During lessons, enable teachers to access Internet resources by clicking on saved image-based 
bookmarks, 

• Allow teachers to access others’ cubes in the hopes of finding already made cubes for a particular 
topic without having to reinvent the wheel for a particular learning activity. 
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Based on these goals, the design team developed a prototype using Adobe Flash Builder (i.e. Flex) and 
Adobe Flash Media Server (i.e. Flash Communication Server). Adobe Flash Builder was used to create the 
environment and Flash Media Server provided the functionality to view and edit a shared cube between and among 
multiple teachers. The current Cube prototype uses a Flickr API (i.e. application programming interface) that allows 
users to search within the Flickr images database. A screenshot of the Cube homepage and search box is provided in 
Figure 1. 




Figure 1: The Cube homepage. This figure provides an example of the Cube grid and search box. 

Once the search is complete, each Flickr image (i.e. clickable visual preview) is available on the search grid. 
For example, in Figure 2, searching for ‘technology’ resulted in the following new search grid on the right side of 
the screen. The teacher can click and drag a resource from the search grid to the blank grid (i.e. the Cube). 
Additionally, if the teacher double-clicked on any Flickr image, the Internet resource would be opened in a new 
window. 
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Figure 2: The grid feature in the Cube. This figure provides an example of the search grid after the user enters 
‘technology’ as a search string. The results are the first thirty results from the Flickr image search. 

Features of the Cube 

The features available in the first prototype of the Cube are listed below and followed by the ‘bolded’ 
proposed features for the second prototype (currently in development). The features include: 

• Images = Flickr API 

• Grid = 5x5 cell, two dimensional rectangular grid of resources 

• Functionality = click and drag of resources 

• Collaboration = editing of shared Cube via Flash Media Server 

• Images = Google API, involves taking screen captures of Internet pages 

• Images = Documents, involves saving thumbnails of documents 

• Grid = 5x5 cell, three dimensional cube grid of resources 

• Functionality = upload documents and other files on hard drive 

• Collaboration = save feature 

• Collaboration = allow teacher to make cube ‘public’ so it can be viewed and shared 

The first prototype uses a Flickr API with image search results, but the goal is to have this work with any 
Internet resource. In the second prototype, text-based results from a Google API will be converted into images using 
a screen capture feature. The system will continue to be a 5 x 5 grid, however, the design team will modify the two 
dimensional rectangle into a three dimensional cube. The design team will add upload and save features to improve 
functionality and collaboration. Users will have the ability to upload a document from a hard drive and the Cube will 
create a thumbnail preview of the document. Finally, the Cube will include a public option when saving. Therefore, 
the Cube can be accessed and used by other teachers who are searching for resources on a similar topic. 

Conclusion / Future Study 

A collaborative image-based bookmarking tool can provide teachers with a shared set of resources. 
Collaboration and click-and-drag features provide educators with the ability to not only work together to complete a 
bookmarking page, but to do so easily and intuitively. The design process explained in this paper is the first step in a 
larger study that will test the use of a collaborative image-based bookmarking tool in an elementary classroom. The 
following questions must be explored to expand this research and contribute to the body of knowledge: Will using 
clickable visual previews as bookmarks increase the speed of teacher recognition of resources? Will elementary’ 
students be more capable of staying on task when teachers use an image-based bookmarking system? In addition to 
these questions, perceptions and attitudes of using the system must be explored. There is much promise for the use 
of an image-based bookmarking system in elementary education and the Cube provides a good starting point. 

References 

Cooper, A., Reimann, R. & Cronin, D. (2007). About face 3: The essentials of interaction design. Indianapolis, IN: 
Wiley Publishing, Inc. 

delicious. (2010). delicious. Retrieved October 10, 2010, from http://www.delicious.com 

Gall, M. D., Gall, J. P., & Borg, W. R. (2003). Educational research: An introduction. Boston, MA: Allyn and Bacon. 

Wepner, S. B., & Tao, L. (2002). From master teacher to master novice: Shifting responsibilities in technology- 
infused classrooms. The Reading Teacher, 55 (7), p. 642-651. 



201 




Video Games: How can their visual conceptual transaction structure be used 

in instruction? 

David Richard Moore 
Patton College of Education 
Ohio University 
United States 
moored3@ohio.edu 

E-Ling Hsiao 

Center for Instruction, Research, and Technology 
Indiana State University 
United States 

E-Ling.Hsiao@indstate.edu 



Descriptors 

Videogame, concept instruction 



Abstract 

Video games have become a multi-billion dollar industry; people of all ages play games on a daily basis. Game play 
is immersive, engaging and some would argue addicting. As a result, educators have long wondered if the medium 
can be used as a tool for education. Unfortunately, the results have been unconvincing and mixed at best. In most 
cases, educational video games are not particularly educational; they are not engaging; and, worst of all, not fun. In 
this presentation I suggest that the fundamental reason for the failure of educational video games results from a 
fundamental misunderstanding of the structure of successful game interaction. A successful video game interaction 
is based upon what I refer to as a "first-order conceptual transaction". Games that provide an opportunity for this 
type of transaction have the potential to be engaging. Learning goals that are congruent with a first-order conceptual 
transaction can successful use a gaming environment for instruction. 

Introduction 

Video games have become popular with all groups of computer users. They are available in arcades, TV 
consoles, home computers and mobile devices. The industry that supports these games has surpassed the motion 
picture industry in terms of its scope and economic impact (Aldrich, 2009). Popidarity along with the many and 
varied types of interactions that can be encountered in these games make them attractive to educators. The 
educator’s desire, of course, is that learners could use the games not only to learn but to enjoy the process as well. 
These two factors, effectiveness and appeal are often missing from educational experiences (Keller, 1987). 

For the purpose of this analysis we are defining video games as computerized games played by 
manipulating images on a video display in response to an algorithm influenced opponent. Key components of games 
are goals, rules, challenge, and interaction. Contrast, this definition with some common, related technologies, such 
as simulations and immersive environments. Simulations are representations, either enhancements or 
simplifications, of realities. They are environments for experiencing aspects of reality. Simulating generally entails 
representing certain key attributes or characteristics of a physical or abstract system. Digital immersive 
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environments are artificial, transactive, computer-created scene or ‘worlds’ within which one can participate. 
Simulations and digital immersive environments have specifically been left out of the current analysis. In these 
systems there is often not, or it is often not a as prominent, a manufactured conflict or pre-established objective as 
there are in games. These definitions are used loosely. There are no clear, unambiguous boundaries. However, we 
will argue that a stable difference can be found in the predominant form in which concepts are represented in each of 
these environments. 

Conceptualization 

Concepts have very specific meaning in instructional design. The term is often used by the general public 
in a sense that is synonymous with notions or ideas. However, instructional designers generally use the term to 
indicate a particular type of learning domain (Merrill, 1983). A concept can be defined as the cognitive integration 
of two or more entities based upon their unifying characteristics (Rand, 1979). Conceptualization is an inductive 
process that requires one to examine specific instances and generalize them to other members of their class. For 
example, bowl, table, liberty, and IQ are concepts. The table that I am typing this is a single instance of the concept 
of table. The concept of table encapsulates every table that ever existed or ever could exist. Conceptualization makes 
it possible for one to use a instance of a concept even if you have never encountered the particular instance. 

Induction through conceptualization is critical to cognition. Induction is a constant and every present in conscious 
experience (Dewey, 1958). Many instructional objectives are concept-based. For example, “distinguish between 
examples of Victorian and Georgian architecture” or “select the examples of art deco furniture from this 
showroom.” Almost any learning objective will require the learning to make discriminations or generalizations 
which are hallmarks of the conceptual learning domain. 

Any element in a game, simulation, or immersive environment is a concept; these entities represent an 
instance of a class. For example, a “gun” on the screen is not just a grouping of pixels but is indicative of the class 
“gun” which has unique attributes and characteristics. Another example might be a mountain on the screen. The 
mountain only vaguely resemble an actual mountain visually, however, the attributes of the mountain will have a 
degree of correlation with the concept of mountain, i.e., the image will be a projection upward to a point relative to 
the surrounding “terrain” and will resist intrusion from other “objects”. Part of gameplay is to recognize these 
entities and react to them according to their class. Transaction with these conceptual entities is the essence of 
everyday experience and is equally so in these environments. 

Teaching concepts 

The process behind and the methods encouraging the learning of concepts should be familiar to anyone in 
the instructional design field. Concepts have been a primary learning domain, Gagne’ (1985), Merrill (1983), 
throughout the history of the instructional design field. The new “Taxonomy of for learning, teaching, and 
assessing” which extends and revises Bloom’s influential taxonomy, has adopted concepts as one of its primary 
taxonomic classifications as well (Anderson & Krathwohl, 2001). 

Concepts are generally taught by declaring their definitions and by providing examples and non-examples 
of entities within the class; examples being the most important part of the instructional process. Concepts are 
generally considered to have a hierarchical nature. Describing the hierarchical nature between a concept and its sub- 
concepts is an effective instructional strategy. It also should be remembered that there is diversity among the kinds 
and types of concepts which have their own unique instructional strategies. For example, concrete concepts (those 
that have a physical manifestation) and abstract ones (those that are completely mental) have different characteristics 
that must be considered when planning instruction. 

It is generally recognized that concrete concepts are easier to learn and grasp. However, most concepts used 
in an academic setting are not concrete, but abstract. In fact, one of the important qualifications for a concept is that 
it does not need to be real in any sense; concepts do not need to have a material existence. However, they must have 
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a function or serve a purpose. The purpose may be trivial but still must be present. For example, a unicorn and a 
leprechaun are concepts that describe things that exist only in fiction; nonetheless they serve a narrative function. 

In fact, the simplest category of a concept is one that simply exists to be classified and reacted to. These 
type of concepts are what we describe as “first-order concepts.” First-order concepts are those that have no apparent 
usefulness beyond their ability to be identified; they have unity and little else. Flowever, their unity qualifies them as 
concepts. A first-order concept’s primary function is in the recognition of the concept itself and not its overt 
functionality or practical usability; being recognized and classified is its function. An example of a first-order 
concept would be “greeny”, which we just made up, meaning everything that manifest the color green. A person 
wearing a green shirt as well as grass and leaves would qualify. One could reasonably ask the purpose of the concept 
“greeny” because it is hard to imagine a real-life situation in which color exclusively would be a useful 
distinguishing trait. Flowever, one could imagine the “greeny” would be useful in an artificial setting, such as a 
game; it would be a simply construct to provoke reaction. 

Concepts in games 

Games require users to react (identify, classify, and respond) to a concept simply because it promotes 
gameplay. Games are the ideal environment for demonstrating the use of a first-order concept because it can be 
created for no other purpose. Its purpose is to be identified, classified and reacted to in such a fashion that 
maximizes the interest of the player. First-order concepts have no need to have a corollary in the real world. All they 
need is to have attributes that can be consistently identified. By using first-order concepts, a game designer can 
create an infinite number of instances in which players can interact with. These instances can be refined to create 
sub-concepts that require even more subtle transactions to maintain the player’s interest. 

A video game will often use names likes ships, monsters, opponents or enemies but those are merely 
names. Their properties are likely to have no corollary with their designations. Usually, these entities are made up of 
unique pixel combinations and audio characteristics. For example, in the game Defender the opponent’s ships are 
merely a conglomeration of green dots on the screen that move in a particular pattern. It is a stretch to call these 
representations ships without guidance from the game itself. The player’s job is to recognize them as opponents and 
then to respond to them (avoid them or shoot them). Note that even “shooting” is only a term in this context. A more 
accurate description would be that you are creating a line of pixels that will interact with the opponents pixels when 
their paths intersect. In other words, the player is not shooting and avoiding enemy ships but identifying an entity as 
a particular concept and reacting and interacting to the presence of that concept. Shooting works as a useful trope in 
a game setting because it allows the player to concisely make their identifications known across the entirety of the 
screen. Some entities in a video game will look like real world objects but this is not necessary for game play. 

What makes first-order concepts so useful in this setting is that they are infinitely malleable in ways that it 
is relatively easy to create the experience of “flow.” Flow is a “peak experience” or an experience that is associated 
with being completely and comprehensively involved and in an activity (Csikszentmihaly, 1990). Flow entails a 
dramatic lowering of inhibition and self-consciousness. In other words, it is a desirable state that is central to the 
entertainment value of video games. Flow is commonly recognized as the element that is pleasant and even 
addicting from an experience. First-order concepts are used, not because they lead to knowledge with utility outside 
the gamespace, but because they are ends in and of themselves. First-order concepts are used by game designers to 
create engagement with the game and not for some other ends. 

One of the primary principles of flow is that a person is continually challenged. The game must continually 
react to the level of conceptualization achieved by the player to create an experience that continually engaging. This 
is called making, “Dynamic Difficulty Adjustments (DDA).” There are a number of methods that have been used to 
implement DDA’s including changing the speed in which an instance of a concept is presented, increasing the 
number of instances which must be responded to, decreasing the time in which an instance must be responded to, 
and decreasing the size of the pixel representation of the instance, among many other options. Perhaps the player is 
asked to make a distinction that they weren’t previously required to perform. Perhaps the entity will have a slightly 
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different shape or color that must be properly classified for game play to continue. First-order concepts are easily 
adjusted and can be adjusted infinitely to maintain a flow experience. 

More advanced techniques would actually modify the concept or split the concept into two or more distinct 
sub-concepts which would require unique transactions with the gamer for successful completion. For example, an 
“enemy ship” on the screen might require a single “blast,” however, an “admiral’s ship” might require two “blasts.” 
The two ships may have only slight visual differences but those must be detected by the gamer. The addition of the 
“admiral’s ship” as a first-order concept provides and opportunity for a DDA at the conceptual level. One of the 
primary advantages of using first-order concepts in games is that the DDA can be endlessly manipulated to maintain 
the gamer’s interest. The reward of gameplay is itself and nothing further. Flowever, many educators have attempted 
to and believe that gameplay can be a means to educational ends. 

Second-order concepts and the constraints of reality 

Contrast the transaction created by the use of first-order concepts to the type of transaction educators are 
interested in. The educator is interested, not in transactions as ends in themselves, but in using them as a means to 
further learning goals. In other words, education is constrained because it is interested in concepts that do have a 
purpose and a function out of the gamespace. Concepts that are required in reality are second-order concepts. 
Second-order concepts are not malleable, they are what they are because the describe entities that must be known, 
classified and identified in real-world transactions. To know them one must respond to them in set and established 
ways and that restricts the degree of freedom allowed in their use and application. 

As it turns out, portraying, interacting and responding to these second-order concepts in gamespace is 
difficult. Learning second-order concepts has many rewards, most prominently mastery and control of their 
prevailing conditions and uses, however, their attainment does not necessarily lead to an experience of flow. Their 
reality simply gets in the way. This is not to suggest that there is a one-to-one correlation between second-order 
status and lack of flow, just that the lack of freedom in their constitution may create natural barriers to their use in 
gamespace. A second-order concept is what it is, and it cannot be modified without damaging its conceptual 
integrity. For example, if one is learning the concept of a “schooner”, a game could be designed that would require 
the learner to recognize examples of schooners in various conditions. Flowever, any display beyond the general 
characteristics and attributes of the “schooner” would have to be of high fidelity (illustrations or photographs). 
Mastery would require a large number of these examples and they would have to be manipulated to integrate with 
the game’s theme, style and motif. First-order concepts, since they don’t require utility or material manifestation are 
restrained in this manner. A second-order concept was made for utility not for being manipulated to provide ideal 
DDA capability. Additionally, once mastery is achieved there is no more possibility of making the event more 
challenging; first-order concepts are not constrained in this manner. 

Further, the difficulty found in second-order concrete concepts is only increased when dealing with second- 
order abstract concepts. For example, if one were trying to teach “democracy” it would be difficult for learners to 
engage with that concept in the same way they engage with first-order concepts. With abstract concepts there is 
simply no visual or auditory representation that can be provided the learner. Abstract concepts are verbal by their 
nature. 

For fear of being reductionist, we want to reiterate that there are game interactions and other similar 
environments like simulations that can indeed address second-order concepts of all kinds. The distinction we make 
in this paper is about the “flow” experience that is so easily generated in game space using first-order concepts. It 
may be possible to create flow using other strategies and tools; however, it is unlikely that many of those will 
possess the kind of efficiency as those that use first-order conceptualization. 

Conclusions 

Second-order concepts are more likely to be found in non-game digital environments. Simulations and 
immersive environments can more readily embraces second-order concepts because they promote a different type of 
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transaction than required for a flow-based game experience. Likewise, there are many types of digital environment 
experiences, such as strategy games, or simulations that can have alternative types of transactions and game play. 
Examples include such environments as SimCity and Civilization. However, these environments are beyond the 
scope of this article in that they are not based upon conceptual transactions. Games, those that are conceptual 
transaction centric, have used first-order concepts to artificially create compelling gameplay. Game designers and 
educators should be cognizant of this distinction in their attempts to create games that teach. It is probable that many 
second-order concepts can be successfully taught through the conceptual transactions provided through gameplay, 
however, it should also be noted, particularly when reports of stilted game play that many second-order concepts are 
simply too restricted in their constitution to be adequately taught in gamespace. Educators attempting to design 
useful games that with second-order concepts should be aware of the inherent disadvantages these types of concepts 
have and seek strategies that mitigate these barriers. 
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Abstract 

The purpose of this study was to explore the use of a dynamic interactive website for computer based 
competency testing in the clinical laboratory. The dynamic interactive website was created using, ColdFusion and 
Dreamweaver. The competency testing for the study was designed to be administered to a group of clinical 
laboratory personnel. For data collection, a Likert survey was designed and administered using the dynamic 
interactive website. The survey was administered to clinical laboratory personnel and instructors. The answers and 
information from the competency testing were synchronously gathered and entered into a Microsoft Access database 
by means of the dynamic interactive website. The results from the survey indicated a positive opinion on using a 
dynamic interactive website. The survey results also indicated that this group of medical laboratory personnel found 
the interactive website effective and useful and preferred using this website for competency testing over the 
traditional written testing format. 

Descriptors: Interactive Website, Computer Based Competency Testing 

Introduction 

The advent of Web 2.0 interactivity has brought about an ongoing effort to define and connect the concepts of 
personal information management and personal information manager. Currently, debate is on Web 3.0 as the 
platform for information management (Laurent, 2010). A personal information manager is a software application 
that keeps track of information. Some common examples of personal information managers are Microsoft Outlook, 
Access database, Excel, Novel Evolution, Lotus Organizer, Kontact, PalmDesktop, etc. Personal information 
management is the study and action of gathering, maintaining, organizing and retrieving information. There are 
different types of manageable information such as printed documents, digital documents, web sites, messages, 
medical records, work-related records, school-related folders, etc. The widespread availability of application such as 
ColdFusion, Dreamweaver and Microsoft Access makes it possible for most people with minimal or no skills in 
computer programming to design their own dynamic interactive website for tracking and managing information. 
Designing and developing an independent interactive website provides autonomy and choice of freedom in deciding 
what and how information is situated on the site. Designing an interactive website offers the ability to maintain and 
control the information gathered from an interactive audience. Additionally, it allows for the collection of 
information to be managed in a private database, information that can later be used for analysis and mining. The 
main advantages of a dynamic interactive website are the increase abilities to track, manage and present data for a 
variety of innovative uses. 

The focus of this study was to explore the use of a dynamic interactive website for computer based 
competency testing in the clinical laboratory. The design of the interactive site for this study was based on several 
premises; concepts of technology integration, learning in a connected world, instructional development, instructional 
design, and open educational resources. 

Research Question 

How do Medical Laboratory Scientists in a clinical laboratory perceive the use of an interactive website for 
computer based competency testing? 
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Literature Review 



The literature review for this paper explored the best educational instructional theories and human 
performance strategies for designing an effective interactive computer base competency testing for use in the clinical 
laboratory departments. The method currently used to assess a technologist's ability to accurately report results for 
both parts of a clinical test is to have the technologist manually perform the first part of the clinical test and then 
review a hand out with pictures illustrating different types of elements found in the microscopic part. The answers 
are then graded by comparing them to results obtained by other technologists and establish criteria. This type of 
assessment is commonly used in most clinical laboratories and doctor’s offices. However, this type of assessment is 
very time consuming in terms of performing, preparing, preserving, and grading. If the test and grading were 
integrated into a computer base program, thus eliminating preparation and preservation of the test samples, the 
employees and supervisors would have more time to devote to actual patient care testing. According to the literature 
review computer based competency testing is more consistent, efficient and effective, compared to the current 
method of competency testing. 

Clinical Laboratories and Competency Testing 

One reason for identifying and surveying interactive competency testing is because clinical laboratories are 
in need of a practical and efficient method for competency testing. The clinical laboratory is an important part of 
the total health care provided to patients. About 70 percent of all medical diagnoses are made using laboratory 
results (Mayo, 2009). The Clinical Laboratory Improvement Act of 1988 (CLIA, 1988) mandates that competency 
testing be done by all employees routinely performing clinical testing at least once a year. Competency testing is 
also required of all personnel working in the clinical laboratories by the College of American Pathologist (CAP) and 
the Joint Commission on Accreditation of Health Organizations (JCAHO). Inspectors for these agencies look for 
proof that competency testing of laboratory personnel are done by two methods: (a) observation and (b) 
documentation. Medical Laboratory Scientists or Medical Technologists must develop and maintain skills necessary 
for interpreting and reporting accurate clinical laboratory test results. Tests in the clinical laboratory often consist of 
two major parts: (a) the manual part and (b) the microscopic part. Integrating computer base competency testing into 
the employee’s routine schedule results in time and resources being better utilized. The current method of 
assessment involves time for performance in addition to time for preparation and preservation of testing samples. 
As part of cost saving, many laboratories are also requiring clinical laboratory personnel, who have been working in 
only one area of the clinical laboratory to retrain as generalists and work in all areas of the laboratory. To determine 
if the laboratory personnel are qualified, employers are using competency testing. Competency includes more than 
just knowledge or information. Competency is the utilization of the knowledge in the actual performance of a 
procedure in a standardized competent manner. Competency is learning how to do something and actually doing it 
right on a day-to-day basis. Written exams can be used to assess the knowledge and direct observation in a real or 
simulated setting can be used to assess competency (Norman, 1991). The goal of competency testing is to measure 
the employee’s level of understanding and provide feedback to the employee and employer on their performance. 
The two types of assessment tests, formative or summative, are conducted. Formative evaluation is typically carried 
out during the development or improvement of a program or product and conducted more than once for in-house 
staff with the intent to improve performance. The results normally remain in-house but are sometimes provided to an 
external evaluator. Many employers are, in an informal sense, constantly doing formative evaluation during their 
observations of workers (Scriven, 1991). In formative testing the results do not contribute to an employee’s grade 
but are used to improve the employee’s knowledge and identify any weaknesses in his or her knowledge and 
understanding of the work. Summative evaluation, on the other hand, is usually quantitative, which means using 
numeric scores or letter grades to assess learner’s or employee’s achievement. 

The subjective nature of reading the microscopic part of a test makes it difficult to develop a consistent and 
valid assessment method. Providing written and consistent grading criteria will help in eliminating the subjectivity 
in reading the microscopic part of a test and also add validity to the method. With the development and 
improvement in computer imaging and technology it is now feasible to use color images of actual microscopic test 
slides. The current method of having the employee review an actual microscopic slide for elements helps with 
improving competency, but the current manual method is time consuming for both the employee and employer. The 
current method of doing it is to have the employee submit a written response and then have them wait for the 
employer to grade it. A computer based competency test will allow the use of digital images of elements for review, 
making the process more efficient both for testing and grading. 
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In a meta-analysis of studies, Computer Based Instruction (CBI) in the health professions was reviewed for 
(a) achievement effects (b) adult learners' long-term retention (c) attitudes toward content and (d) instructional 
method. The analysis consisted of 47 studies with 10 classified as allied health studies. In 32 of the studies, the 
students using CBI had higher examination averages (Cohen & Dacanay, 1992). Bruce (1990) found that in clinical 
laboratory science, computer-assisted instruction is more effective than a correspondence course and as effective as 
a workshop in providing continuing education. Computer-aided instruction provides a pleasant variation in teaching 
techniques, saves instructors time, and appears to stimulate active student discussion. 

CBI draws from several disciplines such as design theory, learning theories, instructional theories. Williams 
(2000) states that there were several different views on what learning theories best applied to learning by means of 
computer based instruction. Behaviorism and constructivism theories have impacted online learning. Currently, most 
courses are being designed along the constructivism theory. In behaviorism the main point of interest is to produce a 
desired behavior that is controlled by the environment. The constructive view is to explore how learning is occurring 
within the environment created. 

The early research studies on computer-based instruction compared it with instructor led instruction. The 
studies tended to view the computer as an independent variable and assumed that the computer itself was affecting 
the learning process (Thompson, Simonson, & Hargrave, 1993). Current research is more focused on the computer 
environment and how it has the potential for improving students problem solving and critical thinking (Krange & 
Ludvigsen, 2008). The new computer environments are promoting the more active and individualized learning on 
the part of the students as well as encouraging teachers to act as facilitators instead of the sage on stage. 

Advantages of computer based instruction includes the ability to reach a much wider audience, the learner's 
control of pacing and approach, flexibility in instructional methodology, and assessment. However, the effectiveness 
of computer based instruction depends on developing proper pedagogical delivery of instruction and testing. 

Some of the alleged disadvantages of computer based instruction are: (a) it is not as effective as having an 
actual person teach. CBI studies do not compare equal teaching methods because the resources do not have the 
pedagogical strategies or knowledge content (Prewitt, 1998) (b) perceived as impersonal and fails to identify 
learners or learning tasks for which CBI may be most appropriate (Lawson, 1999) (c) there is no significant 
difference between the level of learning in computer based instruction and the traditional or standard method. CBI 
users usually devote more time to the common learning goal than their peers so any improvement in performance 
was due to more effort and not something inherently better in using computers to teach (Williams & Zahed, 2005). 

Use of Online Surveys 

The use of computers for surveys is fairly new and is still developing. At first creating and administering on- 
line surveys was very time consuming. It required skills in computer programming and scripting programs. 
Nowadays we have survey authoring software packages and online survey services such as Survey Monkey and 
Zoomerang. The main problem now is that practitioners in the different disciplines do not know all the advantages 
and disadvantages associated with conducting on-line surveys for their unique and particular area. 

The introduction of the Internet and e-mail opened up a new avenue for conducting surveys. Instead of 
providing the respondent with a paper and pencil survey, a respondent is now given a hyperlink to a Website survey. 
In an e-mail survey the questionnaire is sent by e-mail, which the respondent fills out and mails back to the 
interviewer. A web survey provides the capabilities of incorporating multi-media graphics and sound into the 
instmment. 

One major advantage of computer-based surveys is that the computer automates the data entry phase of a 
survey. This increases the accuracy of data entry and saves times and cost in collating. This advantage is also being 
utilized in paper surveys by using scannable surveys. In scannable surveys the respondent completes the specially 
designed form by filling in a circle, oval, or square to show his answer. An optical scanner is then used to record the 
responses that are then digitized and entered into a computer for easy manipulation. 
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Mode of Inquiry 



This was a descriptive quantitative research study utilizing a Likert survey to gather data. The participants 
representing a larger population were surveyed only once for this study. The selection of the participants was not 
random, therefore making the process a non probability method. The independent variable was the use of a dynamic 
interactive computer base site for competency testing and the depended variable was the opinion on efficacy, 
preference and usefulness of an interactive website for computer based competency testing and information 
management in the clinical laboratory. 

The main reason for conducting a computer base survey for this study was the convenience and easy of using 
computer base surveys. The survey for this study was developed using the applications: ColdFusion, Dreamweaver 
and Microsoft Access. The reason for designing and developing the survey instead of using one of the many free 
survey applications available on the internet was personal preference for full control of the final product and the 
avoidance of relying on generic commercial products. Furthermore, having full control of a website and database 
encourages creativity and versatility. 

Technologies on how we communicate have contributed and made an impact on how we collect data. The 
data from the survey will be synchronously gathered into an access database once the respondent submits the 
answers. Because of time constraints the computer base survey was the most appropriate instrument for this study. 
The multidimensional construct of this survey is the opinion of clinical laboratory personnel on the use of an 
interactive website in the clinical laboratory for information management and clinical competency testing. The 
opinion is a concept or abstract that is not directly observable or measurable. The three measured dimensions of the 
construct used in this study were (a) efficacy (b) preference and (d) usefulness. The survey was a non-testable Likert 
scale used to measure the opinions of the respondents. The scale of measurement was ordinal; measurements were 
not limited to just label items but the underlying order of responses. The opinions provided insight on the judgment 
or attitude as perceived by the respondents. Italic lettering was used in the survey items to emphasize whether the 
direction of the question was positively or negatively stated (Cox & Cox, 2008). The survey was cross-sectional 
intended to collect data from a sample that was drawn from a predetermined population and collected at one point in 
time. The questions selected were based on personal preference and the literature review on computer based surveys. 
Several of the questions selected were developed by (McVay, 2001) which are designed to measure comfort with 
some of the basic skills and components of online learning and testing. In addition, questions from a study that 
measured the “comfort with online learning” and “self-management of learning” along with the desire for 
interactivity and effectiveness of online learning were also used (Bures, Abrami, & Amundsen, 2000). In total, the 
questionnaire contained 52 Likert items that reflect on the attitude object using a 5-point scale ranging from 
“strongly agree to strongly disagree” with "5 indicating strongly agree and 1 strongly disagree". Included in the 
survey were reverse questions to cancel out the bias of wanting to agree with everything and increase internal 
validity. A 5-point scale was used because the majority of the articles reviewed for this study suggested using a 5- 
point scale to get opinions that are strongly positive, negative or neutral. The answers to the Likert items also were 
used to develop an index score that indicated direction and intensity. Some demographic information was collected 
using an additional six questions. 

The sample for the survey consisted of a convenient sample of 22 persons who work in clinical laboratories. 
The sample was a non probability sample that relied on available participants. The population selected for the study 
was individuals who supervise or had instructional technology experience in the clinical laboratory. The selection 
preference was geared towards the solicitation of opinions from individuals who would benefit from a dynamic 
interactive website utilized for competency testing and information management. Of the 22 persons that were asked 
to participate by e-mail, 20 filled out the survey. It was a 91percent rate of participation which is higher than the 
average of 50 and is considered good. However, this was probably due to sample size and convenient sampling. 
Included with the request for participation was a monetary inducement in the form of a McDonald's 10 dollar gift 
card. 



The characteristics of the respondents that participated in the survey lend themselves to computer based 
surveying. The majority of the individuals work with computers on a daily bases. Most of the respondents are well 
educated and have at least a bachelor’s degree and are familiar with using the internet. The participants also had 
access to the internet either at work or home. Since the survey was limited to gathering data and opinions on a 
dynamic interactive website for competency testing and information management, it was unlikely that 
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confidentiality would be a major issue. The data from the survey was synchronously gathered into a secured 
database once the respondent submitted their answers. 

Data Analysis 

Validity is the truthfulness of the survey or whether it measures what you intent to measure. To check for 
validity of the data collected by the survey an alignment check was conducted as demonstrated by (Cox & Cox, 
2008). A matrix was designed in which the guiding questions that were answered occupy one dimension and the 
questionnaire items the other. Each item was evaluated and checked against the guiding questions each item referred 
to. Afterwards, validity was checked by cross-referencing the content of the instrument elements with similar 
elements discussed in the literature review. Further review of the data indicated that content validity was established. 
The dimensions sampled by the items appeared to be representative of the attribute opinion. Furthermore, the 
correlations between the dimensions appeared low but strong between the items within. 

Table 1 



Sample Alignment Check 
Questions 





1 


2 


3 


4 


5 


6 


7 


8 


9 


Dimension one 






X 






X 






X 


Dimension two 


X 












X 






Dimension three 




X 






X 










Dimension four 








X 








X 





Reliability is consistency in measuring what you want to measure. Validity on the other hand is measuring 
what you intent to measure and implies reliability but reliability does not necessarily implies validity. As part of 
checking for reliability a descriptive analysis of the data and coefficient alpha was done to check for outliers, 
missing data, or any other abnormality that would interfere with the data analysis. The descriptive analysis of the 
data did not show any missing data or outliers. 

Assessing Scale Reliability with Coefficient Alpha 

The statistical software program used to conduct the data analysis in this study was PASW Statistics 
18. An item-by-item analysis was performed to determine if coefficient alpha could be improved by removing 
items. For this analysis all items comprised a good scale so removing items was not necessary. 

The Coefficient alpha calculated for all the 52 items on the construct was .943 indicting good scale reliability 
since it is equal or higher than .70 (Cronbach, 1951). The Coefficient alpha calculated for the dimension represented 
by items 1-13 was .793. The Coefficient alpha calculated for the dimension represented by items 14-26 was .734. 

The Coefficient alpha calculated for the dimension represented by items 27-40 was .968. The Coefficient alpha 
calculated for the dimension represented by items 40-52 was .849. 

The Pearson correlation coefficient between all pairs of questions demonstrated that all the questions 
correlate fairly well so there was no need to consider eliminating any question. A scan of the correlation matrix to 
check the pattern of relationships was conducted. A review of the correlation coefficients did not indicate any values 
greater the 0.9. Correlated coefficients values greater than 0.9 may indicate singularity of the data which could 
result in values being highly correlated, causing problems with factor analysis. 

As part of the study a communality analysis was performed on the data. The communality measures the 
percent to variance in a given variable explained by all the factors jointly and may be interpreted as the reliability of 
the indicator. In general, communalities show for which measured variables the factor analysis is working best and 
least well. The percent of variance for each indicator in this study did not show the need to eliminate any of them. 

A principal component analysis was performed on the data to analyze for any coherent factor structure 
underlying the items on the survey. The component matrix of the principal component analysis produced the factor 
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loadings results for the study. This was the central output for factor analysis. The factor loadings or component 
loadings are the correlation coefficients between the variables (rows) and factors (columns). Factor loadings are the 
basis for imputing a label to the different factors. Loadings above .6 are usually considered "high" and those below 
.4 are "low." Therefore a high loading corresponds to the type of factor and a low does not. As part of the factor 
analysis a rotated factor component matrix was also done to achieve a better spread on the loading of the items. The 
rotated factor matrix analyses yield six factors with Eigen value greater than 2.0 (ranging from 2.175 to 21.352). 
These six factors accounted for 78.229 of the overall variance. After examining the factor loading of the six factors 
it was decided to use only the first three factors with the highest number of items loading above 0.40 since these 
were the factors that had the items evenly spread and appeared to cover the majority of coherent factor structure 
underlying the survey. The three factor selected were (a) efficacy, (b) perception and (c) usefulness. 

Table 2 



Scree Plot 




The scree plot of SPSS data analysis indicated a point of inflexion on the curve. The curve began to tail off 
between 3 and 4 factors. It also tail off again after 5 factors. It was decided that for this study the retention of 3 
factors would be appropriate. 



Results 

Analysis of the data indicated that certain items that grouped together exhibited high loading on the same 
factor. It was concluded that these items measured the same construct. Three dimensions (efficacy, perception, and 
usefulness) were selected based on loading after rotating factor analysis and the resulting reverse coding clusters. 
Although some items cross loaded the majority exhibited high loading on only one factor. The original dimensions 
of perception and worthiness were combined into the dimension of usefulness. The basis for combining the two 
dimensions was after examining the data, a pattern emerge that cluster the questions of perception and worthiness 
into the dimension of usefulness. Further review of the items indicated that by eliminating and modifying some of 
the questions on the survey a better set of data could be gathered. Further review of the data also indicated that the 
goal of data reduction, description of relationship between items and the test theory about the relationship of the 
dimensions were met. 



Discussion and Recommendations 

The clinical laboratories are in need of a comprehensive, effective, and efficient computer base competency 
test to meet the requirements of the accreditation agencies. The literature review indicates that the best methods for 
evaluating and designing an instructional program, such as a computer-based competency test is to first start with a 
viable design that is eclectic. The best method to accomplish this goal is to start by evaluating the design of the 
competency test in a holistic manner utilizing educational theories such as constuctivism. Furthermore, the literature 
review indicates that the advantages of using computer based instruction outweigh the perceived disadvantages. 
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The computer application used to set up the interactive site and survey were Dreamweaver, ColdFusion and 
Microsoft’s Access database. These applications were selected because they provide a practical and convenient 
method of setting up an interactive information management environment on a local or remote server. This 
capability allows the gathering of data for analysis, collating and grading. The website also facilitated the sending 
of invitations for participation by e-mail. Twenty-two persons working in the clinical laboratory were invited to take 
the survey and were directed to the URL of the functional interactive website set up for competency testing. 

The results from the survey indicated that the participants felt using dynamic interactive website was 
preferred, effective and useful for competency testing in the clinical laboratory. In addition the study indicates 
that research to compare current methods of competency testing and computer based testing is needed. Furthermore, 
the study also indicates that research on the different ways of using an interactive website for a variety of 
applications such as on-line surveys, data gathering, local network location, educational instructional design, etc. is 
also needed. 
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Transitioning from Face-to-Face to Online Learning: A Case Study 

Renee Pilbeam 
Angela Barrus 

Options for online learning are becoming increasingly more available, feasible, and possible for many 
students pursuing higher education. The increased use of online learning is partly due to the economic reality of 
shrinking budgets and the increased need of adult learners to pursue professional continuing education 
(Gunawardena and Mclsaac, 1996). With more and more educational institutions offering such options, it is 
important to explore the decisions made by instructional designers when transitioning from face-to-face to online 
formats. 

The structure of a course is an important consideration in designing an online learning environment; it is 
also imperative to student satisfaction and achievement. According to Stein (2004), the most important factor in 
student satisfaction and community formation is the degree of structure in the course. These findings are in contrast 
to transactional theory that states that low structure, the ability to negotiate with the instructor, and the autonomy 
that online learning offers, are most valued by students. Structure includes such things as clearly defined objectives, 
assignments, deadlines, and encouraging dialogue (Stein, 2004). 

Another study conducted by Ausburn (2004) confirms and expands these findings. In this study students were 
asked to rank order eight features of the online learning environments. The results identified the top three features 
as being (1) Course announcements and reminders from the instructor; (2) Course information documents such as 
syllabus, outlines, requirements, and grading procedures and; (3) Information about specific assignments and 
instructions for completing them. An important commonality worth highlighting in each of these top three choices 
is their function as creators of structure and security (Ausburn, 2004). All three of these features accomplish the 
task of helping students stay on-task and focused through guidance and confirmation as they progress through a 
course that is at a distance, which has been shown as a critical component of learner success in distance education. 

In an article entitled “The Ideal Online Course” (Carr-Chellman, 2000), various features of online courses are 
reviewed in the context of how they contribute to creating the ideal in online courses. One such feature explored is 
the use of peer reviews. Encouraging students to post their work online to make them available for review by their 
peers provides for an opportunity to learn from their peers in refining their own work and also encourages a high 
level of intellectual discourse over the approaches and results others have undertaken. 

According to Moore and Kearsley (2005) “Distance education is planned learning that normally occurs in a 
different place from teaching, requiring special course design and instruction techniques, communication through 
various technologies, and special organizational and administrative arrangements” (p. 2). Each of the components of 
this definition has been explored in the current study. The construction of the course also closely addressed the 
elements of multimedia instruction set forth by Martin and Klein (2008) by providing objectives, information, 
practice with feedback, and review in each learning mission. These elements were taken into immediate 
consideration when designing the course and its structure. In conducting the current study by analyzing so many 
aspects of the course, it is hoped that pertinent recommendations can be drawn and contribute to the current body of 
instructional technology literature and practice. 

This study researched the design and development of the course (Richey & Klein, 2007) to examine the 
analysis, design, development, implementation, and evaluation (ADDIE) of a course as it transitions from face-to- 
face to a fully online environment. The study addresses the following research questions: 

1 . What were the design decisions at each stage of the ADDIE model in the execution of the course, and what 
were the justifications and implications accompanying those decisions? 

2. What were student’s attitudes toward the course and its structure, components, and execution? 

3. How did the course design, development, implementation, and evaluation change the instructor/designer’s 
attitude toward the course and teaching it? 

4. Did student achievement vary between the face-to-face and online course? 

The independent variable in this study is course format. One level of this variable is the face-to-face course 
conducted in Spring 2009. The other level is the online course that was designed in Spring 2009, and then 
conducted over Summer Session 2009 and Fall 2009 semesters. From the Spring 2009 running of the face-to-face 
course to the development and running of Summer Session of the course changes were made to the content and 
organization of the course. 

The criterion variables are student attitudes, designer/instructor attitudes, and student achievement. Student 
attitudes have been measured across all levels of the independent variable using a survey format with mostly Likert- 
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scale type questions, and a few open-ended questions. Designer/instructor attitudes were measured through self- 
noting by the two instructor/designers and also apparent in course modifications. Student achievement was 
measured through overall grades in the course and also grades on comparable projects. 

Also of particular importance in this study are the design decisions of the instructors/designers of the course. 
Both of the people involved in the transition process of this course were teaching during the Spring 2009 semester in 
the face-to-face environment with one proceeding on to teach Summer Session in the online environment. 
Elaboration about the design decisions with recommendations at each stage of the ADDIE model are included below 
in the results and discussion. 



Method 



Participants 

Participants are approximately 125 undergraduate students enrolled in a Computer Literacy course at a 
large university in the southwest United States. The instructors and also designers of the course are two Teaching 
Assistants that had taught the course for at least a year and are also doctoral students in an Educational Technology 
program. The terminal goal of the course is for students to become computer literate. The course has two areas of 
concentration: (1) discuss issues surrounding computers, software and the use of technology in the classroom and 
workplace fluently, and (2) use computer applications for productivity, data analysis, and problem solving. Material 
for both versions of the course is hosted through the university’s BlackBoard site. The online version is arranged 
using a gaming format into five “learning missions” that each have six levels to complete before moving to the next 
mission. There was also a group mission, midterm mission, and final mission. 

Procedures 

Two different treatments are explored in this study. The first is the spring semester course in its face-to- 
face format, and the second is the summer and fall semesters in their online format. Assignments, course objectives, 
projects, goals, and the course textbook remained the same for both treatments. Participants were assigned to each 
treatment group through their chosen enrollment in the course. 

The materials used and covered in each course were the same. There were 4 main units in the course. 

Word, PowerPoint, Excel, and ePortfolio (Google sites). Each unit included a chapter reading, discussion, a 
demonstration, a practice activity, a project, and a quiz. There was also a midterm and final exam as part of the 
course. In both the Spring 2009 face-to-face course and the Fall 2009 online course, the each unit occurred over 4-5 
weeks, with the duration of the entire semester lasting 15 weeks. In the Summer 2009 Course the entire duration of 
the course was 5 weeks, with each unit occurring over 1-2 weeks. 

In each treatment students progressed through the course with the help of the instructor and through the 
platform of the BlackBoard Course Management System (BB). Each individual instructor, using rubrics, 
administered grades for discussions, activities, and projects. Quizzes were administered through BB and were 
scored automatically. The instructor using a rubric also scored both the Midterm and Final. For the online courses, 
the demonstrations were done using an internet based Microsoft resource that included pertinent features to the 
course, while in the face-to-face course the demonstrations were done by the instructor. 

Materials 

The notable difference in the two course designs was the organization of the course. In the face-to-face 
version of the course units were organized into “Learning Modules” and further grouped by week numbers with the 
content to be completed that week. In the online version each unit is organized into a “Learning Missions” and 
arranged further in “Levels” in numeric order pertaining to the order to be completed. Content in each was varied 
only minimally only to recreate the demonstration in the online environment without an instructor to deliver that 
live. 

The online environment also utilized the use of a course calendar that was embedded on the announcements 
page in the course BlackBoard site. This was the first thing students viewed when entering the course and it 
included unit start dates, suggested pacing, and assignment due dates. 

Criterion Measures 

The criterion variables in this study are student attitudes, designer/instructor attitudes, and student 
achievement. Student attitudes were measure through a 10-questions survey including eight Likert-type questions 
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about various course attributes, and two open-ended questions allowing participants to give individual feedback. 

The tool used for this measurement was survey monkey. 

Designer/Instructor attitudes were noted during the transition process from face-to-face into online, and 
also during the semesters in which they instructed the course. With the necessity for an online offering of the course 
in future semesters it was decided that the course would be evaluated, revised, and reorganized to go online in July 
2009. During this process the two designer/instructors applied the ADDIE model to efficiently and effectively 
transition the course to online. 

Student achievement was measured via scores on assignments, quizzes, exams, and projects. The criteria 
for grading these assignments were consistent through each semester and were adjusted only to incorporate different 
features of the content taught. Data was archived from Spring 2009, Summer 2009, and Fall 2009 versions of the 
course to be analyzed. 

Results 

The survey used in this case study was designed and delivered in Survey Monkey. Students were allowed 
to participate on a voluntary basis and were asked to answer eight Likert-type questions and two open-ended 
questions. The results of the survey are displayed in tables 1 through 8. Table 9 displays the final course grade 
distribution for students in each of the groups. 

Table 1 

Survey Question 1 - Had you used Microsoft Office 2007 prior to the start of this course? 





Yes 


No 


n 


Summer 2009 


66.7% 


33.3% 


22 


Fall 2009 


76.9% 


23.1% 


78 



Table 2 

Survey Question 2 - Rate your level of proficiency with productivity software prior to starting this course. 
Expert Above Average Average Below Average No Experience n 



Summer 2009 0% 22.7% 50.0% 22.7% 4.5% 22 



Fall 2009 2.5% 40.5% 48.1% 8.9% 0% 79 
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Table 3 

Survey Question 3 - In general how satisfied are you with EDT 321? 





Very Satisfied 


Satisfied 


Neither 


Not Particularly 
Satisfied 


Not Satisfied 


n 


Summer 2009 


36.4% 


50.0% 


9.1% 


4.5% 


0% 


22 


Fall 2009 


34.2% 


35.4% 


12.7% 


11.4% 


6.3% 


79 



Table 4 

Survey Question 4 - How effective are the course practice activities in helping you to learn how to use the 
productivity software? 





Very 

Effective 


Somewhat 

Effective 


Neither 


Somewhat Not 
Effective 


Not Effective At 

All 


n 


Summer 


59.1% 


36.4% 


4.5% 


0.0% 


0.0% 


22 


2009 














Fall 2009 


36.7% 


54.4% 


3.8% 


3.8% 


1.3% 


79 



Table 5 

Survey Questions 5 - The instruction has prepared me to successfully complete the project for each Learning 
Mission. 

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree n 



Summer 2009 45.5% 50.0% 4.5% 0.0% 0.0% 22 



Fall 2009 32.9% 43.0% 12.7% 5.1% 6.3% 79 
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Table 6 

Survey Question 6 - The arrangement of the course into learning missions and levels made it easy to progress 
through. 

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree n 

Summer 2009 50.0% 45.5% 4.5% 0.0% 0.0% 22 

Fall 2009 42.9% 45.5% 6.5% 2.6% 2.6% 77 

Table 7 

Survey Question 7 - The course calendar was useful to have access to throughout the course. 

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree n 

Summer 2009 54.5% 45.5% 0.0% 0.0% 0.0% 22 

Fall 2009 67.9% 25.6% 2.6% 1.3% 2.6% 78 

Table 8 

Survey Question 8 - The use of peer reviews in the course were helpful in my understanding of the project 
requirements. 

Strongly Agree Agree Neither Agree nor Disagree Disagree Strongly Disagree n 

Summer 2009 22.7% 27.3% 36.4% 9.1% 4.5% 22 

Fall 2009 16.9% 35.1% 16.9% 22.1% 9.1% 77 
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Table 9 

Student Achievement Data - Overall final course grade distribution. 

A B C D E I W n 



Spring 2009 73.6% 13.9% 2.8% 1.4% 2.8% 0.0% 5.6% 72 



Summer 2009 64.0% 16.0% 4.0% 0.0% 0.0% 0.0% 16.0% 25 



Fall 2009 51.2% 19.0% 10.7% 0.8% 7.4% 0.8% 9.9% 121 



There were also two open-ended questions that were answered by 21 of the students in the summer course 
and 68 in the fall course. A sampling of the responses is below for each question. 

Question 9 - What did you find most enjoyable and/or most useful about the course? 

• The tools that this course taught and provided are very important in society today, and I am grateful to have 
been able to know how to use them through this course! 

• I really liked how each section of the course was split into learning missions and different levels. Since the 
course covers so much material 1 felt that this helped me to manage time and make it seem less daunting. I 
thought that this in addition to the course calendar that had pacing dates was extremely helpful and useful 
and was one of the reasons I was able to succeed so well in this course. 

• I enjoyed how we had just basic guidelines and we could use our interests and imaginations while creating 
projects. 

• The course calendar. 

• Class was great! It required a lot of effort, which is expected but it has been the best online class I have 
taken. It was easy to know what should be done with the calendar and if you did things that were asked of 
you, you did well in the class! 

• I really appreciated the set up of the course; the way it was separated into the "Learning Missions" made it 
very easy to follow along. I enjoyed making the Google Site the most. 

• The tool practices, it gave me a chance to practice the skills before the test. 

• The lay out of the missions and the levels. 

• I liked that we were shown some new software and were given projects to help us understand the software. 
The projects were a new learning experience as well as fun to do. The calendar that showed due dates as 
well as kept you on track. Great Idea, and it was the first page that you saw when you loaded blackboard. 

• There were fun assignments that tapped into my creativity! 

• 1 liked the PowerPoint mission and the group projects. It was fun to create a presentation and to learn how 
to use all the cool features of PowerPoint. 

• The organization of the missions and the tutorials. 

• 1 thought the lessons where they walked you through with instructions of how to use 2007; excel, word and 
PowerPoint were very helpful. 1 learned new tricks of how to manipulate the program to work to my 
advantage. 

• The calendar, Great way to stay on track. 

• The way the online course was designed was very enjoyable and useful. It made it easy to follow and figure 
out a daily basis what was expected of me, what assignments I needed to do, the deadlines, everything was 
great! 
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• 1 found doing the tool projects most enjoyable because I was able to be creative. 

• 1 truly enjoyed taking this class in many different ways. I thought the peer reviews were very useful. I 

thought the class was well organized and the lessons were useful. The videos were also very helpful. 

Question 10 - What suggestions do you have to improve and/or change the online EDT 321 course in the future? 

• I enjoy doing peer reviews but I wasn't sure 1 liked how we were assigned groups. While I understand that 

this was done so everyone had their project reviewed, I felt it was hard to communicate with some people 
because they may not have been checking blackboard as often as other people. Perhaps more leniency on 
which we can peer review would help. 

• Gear it more towards the diverse population of students. I have no intention to be a teacher, but it seemed 
every activity related to that profession. Explore more software. Other software exists outside of Microsoft. 

• My only suggestions might be to have the projects graded sooner and maybe put more of an explanation 
under the project instructions, because some of the projects 1 had no idea where to start and didn't know if I 
was doing it correctly. Other than that, good! 1 learned a lot. 

• 1 felt like one in a thousand students in this course. The instructor I had was short with me in my emails 
about questions and did not ever take the time to fully explain what I was asking. Several of my emails 
were never even responded to. The peer reviews often left me no closer to understanding the guidelines 
because my peers would not take the time to actually analyze my project. Also the group project was 
extremely frustrating and seemed inappropriate for an online course. I did most of the work and everyone 
got credit for it. The MyLabSchool section of the course was a waste of my money. We used the website 
maybe four times and it did not help me to better understand the course materials. 

• Make instructors more easily accessible and also timelier responses to emails in the future. 

• I did like that the learning mission presentations were open to us to choose our own topics, but I would 
have loved to see examples for the level 5 learning missions. Since the instructions sometimes were so 
broad, I would have like to see an example of our expectations so we knew we were on the right track. 
Great course, though! 

• I didn't care for the online book. It was expensive and as I understand ASU will not buy back these types of 
books. I would suggest something that will make it less costly for the students and with resale ability. 

• A new textbook... The one we have now seems outdated. 

• I am not an education major and really struggled with many of the assignments that were geared towards 
EDU majors. For example, I had no idea how to write a lesson plan and think that you should have an 
alternative assignment for those of us that are not seeking careers in the field of education. 

• Better instructions and feedback from the teachers.... also grade our assignments quicker. 

• Unfortunately, I didn't receive strong feedback or help on the peer review. 1 think this should be deleted. 1 
think if students have questions of not understanding the material. It should be aired in the discussion board 
and not through email. I received too many emails from students as they "Replied all" and did not include 
the instructor. Going through the discussion board will include the instructor. 

Discussion 

This study aimed to explore the design and development of a course that was a face-to-face course and was 
transitioned to an online course. In its case study design that included student’s surveys, instructor impressions, and 
student achievement data, the course itself and the success of the transition process were analyzed. Overall, the 
transition seems a success with many favorable ratings and pieces of feedback from students, the instructors, and 
designers. 

Emerging as one of the most interesting features of the course in its redesign and launching in the Summer 
and Fall semesters was the organization of the material into “Learning Missions” with six “Levels” to complete in 
each unit before moving to the next. Table 6 shows the results to this question on the survey and there is a favorable 
(strongly agree or agree) rating from 95.5% of students in the summer course and 88.4% from students in the fall 
course. The intent of the designers in organizing and naming the content this way was to make it seem more like a 
game, be less formal in it’s naming, and be very easy to follow through knowing the order of completion for tasks. 
From the results in the survey this seems to have been achieved. Also, in the open-ended question (question 9) 
students often commented on their affinity for the organization of the course and mentioned the terminology of 
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missions and levels in doing so. This implies that it both was easily understood and also stuck with them throughout 
the course. 

The most surprising finding of the study was how the peer reviews faired. It was hypothesized that the 
students would be very fond of the peer reviews, find them helpful in learning the criteria they would be assessed 
upon, and be an effective form of feedback. In the survey they were rated favorably by 50.0% of students and in the 
fall by 52.0% of students. They were also mentioned often in the open-ended question directed at improvements 
that could be made to the course. Possible reasons for this could be the way in which they were conducted via the 
discussion board in BlackBoard may not have been the best tool for an activity such as this. Also, especially in the 
summer course the time constraints could have created an unfavorable opinion because of some students 
unresponsiveness affecting others ability to complete the peer reviews. 

Another question that addressed a new feature in the online course and yielded valuable results was that 
focusing on the course calendar. In creating the course calendar the instructor/designer intentions were to create a 
very clear, concise, and easily accessible overview of the course and it’s important due dates. Google calendars was 
used and embedded because of it’s accessibility by all students at the university in coming with their accounts and 
also because of it’s abilities to be updated easily with those changes immediately available to students to view. In 
the survey questions presented in table 6, students in the summer course gave the course calendar a favorable rating 
99.0% of the time and in the fall they did so 93.5% of the time. Some suggestions for improvement included 
making the course calendar a part of the syllabus rather than as a separate web-based resource, but in large part 
feedback about the calendar was quite favorable. 

The importance of practice was also assessed in the questionnaire to students. Presented in table 4 results 
indicated that in the summer course 95.5% of students said the course practice activities were either very effective of 
effective in helping them to learn the productivity software. In the fall course 91.1% felt the same. 

Student Achievement was measured through the comparison of the overall course grade distribution from 
the face-to-face sections of Spring 2009 to the online sections of Summer 2009 and Fall 2009. The same instructor 
taught these sections. The important points to note here are that the percentage of grades that were an “A” were 
higher in the face-to-face section than either of the online sections. The percentage of participants receiving a 
course grade of “B” or “C” were less in the face-to-face than either online section. The number of course 
withdrawals, “W”, was also lowest in the face-to-face sections of the course. The only grade which was assigned 
more in the face-to-face course was the grade of a “D”. Overall, the general distribution of the grades in each of the 
sections did not vary greatly and followed the same general pattern. 

The impressions of the instructors/designers were very positive toward the transition of the course from the 
face-to-face format to online. The transition took place over approximately 3 weeks where the designers met often 
and built the course in BB. Much attention was paid to creating the “game-like” structure with the materials 
organized into “Missions” with “Levels” to be completed. The designers were also careful to include in each 
mission the objectives, practice, and review. The assessment pieces were only modified slightly to fit the online 
format. The overall transition process was smooth and successful from the designer perspective and subsequently 
from the instructor perspective. 

Throughout the process of designing this course developing it, teaching it, and collecting and analyzing 
data several suggestions for future research have surfaced. The one seemingly most exciting and current with the 
times is the idea of creating a course around a “game-like” structure. Using different models for organization of the 
material, naming schemes and activities might be worthwhile to explore in the future. Another is experimenting 
with alternate tools or ways to conduct peer reviews in an online environment to increase student satisfaction in 
getting feedback from peers. A final recommendation here stemming from this research is the idea of building the 
course calendar into the syllabus creating more of a study guide in a static form rather than the dynamic web-based 
route pursued here. It is of course important to keep in mind the limited constrains of this study in it’s number of 
participants, specific audience and content, and information gathered as one proceeds on. 
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Effect of Academic Incentive Use with Cooperative Learning Groups 
on Concept Acquisition and Attitudes 

Rim Razzouk 
Florida State University 

Abstract 

The purpose of this study was to examine the effect of academic incentive use with cooperative 
learning groups on concept acquisition, and attitudes towards instruction in a classroom 
assessment course. Forty-four undergraduate students from two sections of the course were 
randomly assigned to one of two treatments. Then, students in each section were randomly 
assigned to cooperative learning groups. Both groups received guidelines about working 
effectively in groups and instructions on writing objective test items. The experimental group 
was offered an academic incentive; however, the control group was not offered any academic 
incentive. Results revealed that students in both groups performed, on average, equally on the 
concept acquisition assessment. But, there was a significant difference between the two groups 
on confidence (p = 0.012). The findings are discussed in terms of cooperative learning elements, 
instructional material design, and achievement goal as factors likely to have led to the results. 



In the last few decades, there has been a growing interest in the use of classroom instructional 
activities, such as cooperative learning, that allow learners to be active participants in their own 
learning rather than passive observers (Johnson & Johnson, 1999; Johnson, Johnson, & Smith, 
1998b). Many authors have defined cooperative learning and its fundamentals to be successful 
(Johnson & Johnson, 1999; Johnson, Johnson, & Smith, 1998b; Larry, 1993; Walker, 1996). 
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Even though the terminologies used to define cooperative learning and its basic elements vary 
among authors, all the descriptions have a set of common ideas. Johnson and Johnson (1999) 
definition of cooperative learning and description for its basic elements capture the major 
thoughts discussed by the other authors. 

Johnson and Johnson (1999) defined cooperative learning as a versatile procedure that 
can be used for diversified purposes. It consists of learners working together (cooperative 
learning groups) to accomplish shared learning goals that are beneficial to all and to complete 
specific tasks and assignments. These groups may be used to teach specific content, to ensure 
active cognitive processing of information during a lecture or demonstration, and to provide 
long-term support and assistance for academic progress. The essential elements that Johnson and 
Johnson described for a successful cooperative learning include (a) positive interdependence, (b) 
individual accountability, (c) face-to-face promotive interaction, (d) social skills, and (e) group 
processing. 

While there have been a number of experimental and quasi-experimental studies that 
have compared the effects of cooperative learning groups with individual learning, results have 
varied greatly across studies. In some instances students within cooperative groups learned more 
and had more positive attitudes toward learning than students who worked individually (Klein & 
Predimore, 1992; Saner, McCaffrey, Stacher, Klein, & Bell 1994). However, in other cases, 
students who worked individually demonstrated greater learning and more positive attitudes than 
those students who worked in cooperative groups (Klein, Erchul, & Pridemore, 1994; Klein & 
Schnakenbery, 2000). Some other studies have also reported that students who worked in 
cooperative groups and students who worked individually were equally effective (Carrier & 
Sales, 1987; Crooks, Klein, Jones, & Dwyer, 1996). Moreover, in those instances in which 
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cooperative learning proved to be superior, the researchers were unable to ascertain which 
elements of the cooperative learning were primarily responsible for the improvement in learning 
and attitudes. 

Given the mixed results and the difficulty of previous researchers have had isolating 
those elements of cooperative learning that may be of particular importance, I thought it would 
be beneficial to conduct a study that focused on academic incentive use with cooperative groups 
to strengthen one particular element of cooperative learning. As previously noted, one such 
element involves positive interdependence. Positive interdependence refers to the cooperation 
when students promote and facilitate each other’s efforts to learn (Johnson, Johnson, & Smith, 
1998). One way to strengthen positive interdependence is by using joint rewards, such as 
academic incentives, where all members of a group receive the same reward based on the group 
performance (Jonhson & Johnson, 1999). An academic incentive is any tangible or intangible 
reward that is designated to motivate the group of learners to achieve a specified learning goal 
(Stipek, 2002). 

There appears to be a little consensus within the instructional design community as to 
how much academic incentives affect concept acquisition. Some theorists have emphasized the 
value of academic incentives in enhancing concept acquisition (e.g., Hay, 1976; Kagan 1992; 
Johnson & Johnson, 1999). Others, however, have argued that academic incentives for group 
work are counterproductive and can worsen learning (e.g., Kohn, 1991). 

Research related to the effects of academic incentives on learning has also shown 
conflicting results. Some researchers have supported the idea that academic incentives do not 
affect learning in cooperative groups (Brewer & Klein, 2006; Klein & Schnackenberg, 2000; 
Neihoff & Mesch, 1990). Regardless, there is some evidence that appropriate academic 
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incentives have a positive impact on learning when learners use cooperative learning methods 
(Cole & Smith, 1993; Jensen, Johnson, & Johnson, 2002; Joyce, 1999; O’Donnell 1996). 

Theorists also appear to have little agreement on the effect of academic incentives on 
learner attitudes. Slavin (1991) asserted that academic incentives are necessary to enhance 
attitude in cooperative group settings. However, Kohn (1991) believed that the academic 
incentives for group work are ineffective and can deteriorate attitude. 

The findings relating to the effects of academic incentives on learner attitudes have also 
been mixed. Some research has supported the notion that academic incentives improve learner 
attitude (Johnson, Johnson, & Hollubec, 1994; Webb, 1997). However, other research has 
indicated that learner attitudes are not affected by the academic incentive in a cooperative group 
(Adams, Carson, & Hamm, 1990; Klein & Schnackenberg, 2000; Johnson & Johnson, 1996). 

Despite the steady growth in the intellectual writing and theorizing around the impact of 
incentives use with cooperative groups on concept acquisition and attitudes, the research in this 
area remains very limited. The previously presented studies provide a little clarity and evidence 
as to the relative effectiveness of incentives within cooperative groups on concept acquisition 
and learner attitudes. 

The purpose of this study was to examine the effect of academic incentive use with 
cooperative learning groups on acquiring concepts of constructing objective test items, and on 
attitudes towards instruction for undergraduate students in an introductory classroom assessment 
course. Specifically, the study compared the experimental and the control group on acquiring the 
concepts of constructing objective test items and their attitudes towards instruction. The 
experimental group included cooperative learning groups who were offered bonus points for 
their post-test grade and each of the individuals grade affected the success of the group as a 
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whole. However, the control group consisted of cooperative learning groups who were not 
offered any bonus points for their post-test grade and their grade did not affect any other students 
in the group to which they belong. 

Concept acquisition was measured through the administration of a post-test that required 
learners to demonstrate their understanding and ability to apply the concepts of constructing 
objective test items. The items on the test measured recalling and applying the concepts. The test 
was administered at the end of the study, that is, after the given instruction and the practiced 
activities in a few days. 

Learners’ attitudes towards instruction were measured through an attitudinal survey based 
on the Keller’s Course Interest Survey (CIS) and the Instructional Materials Motivation Survey 
(IMMS) at the end of the activities practice. Students’ attitudes were measured in tenns of 
attention, relevance, confidence, and satisfaction towards instruction and material. The goal with 
this instrument is to find out how motivated students were by the lesson (Keller, 2010). This 
survey required students to report their degree of agreement with variety of different statements 
concerning their attention, relevance, confidence, and satisfaction. 

In this study, I hypothesized that undergraduate students in cooperative learning groups 
that were offered an academic incentive would do better on recalling and applying concepts of 
constructing objective test items than undergraduate students in cooperative learning groups who 
were not offered an academic incentive. This result was expected based on the positive 
interdependence concept that links the success of the whole group to the individual contributions 
that are necessary to reach an objective or receive a reward (Johnson & Johnson, 1999). In 
addition to this concept, the positive reinforcement theory states that a satisfying positive 
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reinforcement (e.g., bonus points) which is contingent upon a response results in strengthening 
that response (Driscoll, 2005). 

I also expected that undergraduate students in cooperative groups that were offered an 
academic incentive would have better attitudes towards instruction than undergraduate students 
in cooperative groups that were not offered an academic incentive. Logically, when a student is 
rewarded for an achievement, he/she feels more motivated, which in turns affects his/her 
attitudes towards the instruction. In support to this logic, Keller believes that students motives, 
together with their expectancies (outcome expectations), influence their attitudes because they do 
not look at rewards in isolation (Driscoll, 2005). 

Method 

Participants 

The participants in this study were 44 undergraduate students who were taking a 
classroom assessment course at the Florida State University. The study took place during the 
regularly scheduled periods of the class. Thirty students were white, 6 were black, 5 were 
Hispanic, and 3 with other ethnic background. The majority of the participants were females. 
Thirty seven students were females, 6 students were male, and 1 of the student’s gender was not 
indicated. Twenty two of the students were seniors, 20 were juniors, 1 sophomore, and 1 
appeared to be a graduate student; even though, the course was an undergraduate course. The 
participants in the study were all the students in the two sections of the classroom assessment 
course in the university. 

Materials 

The same instructional materials and tasks were given to both sections. All students were 
provided with paper-based guidelines about how to work effectively and efficiently in a team. 
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These guidelines were distributed by the teacher of every section in the class prior to the 
instruction session. The students were also supposed to read a chapter that focuses on concepts 
for constructing objective test items before the following class session. During the instruction 
session, the instructors used PowerPoint presentation to deliver the instruction focusing on 
constructing objective test items. The length of the instruction was for around 45 minutes and its 
purpose was to teach students the concepts for building objective test items. After the instruction, 
the students worked in cooperative learning groups on a 20 minutes paper-based practice in the 
classroom to apply the concepts that were covered by the instructors. The intervention (i.e., the 
academic incentive) was employed in this part of the instruction. The students were allowed to 
use their books to refer to information from the chapter to stimulate recalling prior infonnation 
and concepts that were presented by the instructors. Each section in the practice provided 
guidance and directions for practicing the specified concept. After the practice, a paper-based 
feedback was provided to students to show them whether or not their answers were correct. 
Independent Variables 

The independent variable was the use of academic incentive with cooperative learning 
groups. The first level of the independent variable was cooperative learning groups who were 
offered an academic incentive (bonus points). The second level of the independent variable was 
cooperative learning groups who were not offered any academic incentive. 

The cooperative learning groups who were offered an academic incentive were told that 
in order to receive the bonus points each student in the team should get 88% or higher in the 
post- test and that the success of the group was based on the perfonnance of each individual. 
Throughout the practice activity in the classroom, I always reminded the students that their 
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success was based on their interdependent perfonnance and encouraged them to work 
cooperatively on the activity. 

The success of cooperative learning groups who were not offered an academic incentive 
was dependent on individual success and not on the other group members’ success. These 
students were not offered any academic incentive. Before the students started working on the 
activity in the classroom, I only encouraged them to work cooperatively within their teams. 

Dependent Variables 

The dependent variables in this study were the learners’ acquisition of constructing 
objective test items concepts, and the learners’ attitudes towards instruction. 

Learners’ acquisition of the concepts was measured through a paper-based post-test. The 
test was designed to assess the learners’ ability to recall and apply the concepts related to 
constructing objective test items. The test consisted of 18 multiple choice questions. Nine 
questions measured the students’ ability to apply the concepts and 9 questions measured their 
ability to recall infonnation. One of the test questions that measured the students’ ability to 
apply concepts was: “Which of the following is the best-stated short answer item?” followed by 

4 choices: A. Cleveland may be found on . B. A person first landed on the moon in 

. C. The author of Silas Marner is . D. The United Nations 

building is in . While one of the questions that measured recalling 

information was: “Which item type is classified as a supply-type item?” followed by 3 choices: 
A. Matching. B. Short answer. C. True-false. The test items were selected from the test-bank 
questions related to the chapter in the instructor manual. The key for the answers from the same 
manual was used to score the students responses. 
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The learners’ attitudes were measured in tenns of attention, relevance, confidence, and 
satisfaction using a survey that included questions from Keller’s Course Interest Survey (CIS) 
and the Instructional Material Motivation Survey (IMMS). The survey was paper-based and 
included 24 items, 5 of which related to attention, 7 related to the relevance of the materials and 
tasks, 7 pertained to learner’s confidence, and 5 items relevant to measuring learner’s 
satisfaction. Responses to the items were in the form of a five-point Likert scale. 

Procedures 

Since the instructors were both willing to give bonus points to their students, the sections 
were randomly assigned to either the treatment group (students who were offered the academic 
incentive) or control group (students who were not offered academic incentive). Then, 
participants in each group were randomly assigned to cooperative groups within their class. The 
instructors were asked to distribute the guidelines for working in teams to the students and to ask 
them to read the “constructing objective test items” chapter prior to the instruction and practice 
session, and to bring their books to class in the following session. Both, the treatment and the 
control groups, participated in the study over two days, dedicating 1 hour and 50 minutes from 
the class time for the whole study. On the first day, the students worked on the practice activities, 
and completed the attitudinal survey after the 45 minute-instruction that was delivered by the 
instructors, and on the second day, the students completed the post-test. Only students who were 
present on both days of the study and for whom data were obtained (n=44) were included in the 
final analysis. 

The study was conducted in the classroom assessment course for the undergraduate 
students with the teacher and the researchers in the class throughout the study. The study for 



232 




each of the groups was conducted on a different day; however, the days were consecutive and the 
time between the first day and the second day of the study was equal for both groups. On the first 
day after the instruction was over, the instructors introduced the research team to the participants 
and the researchers in return informed the students about the purpose of the study, and described 
for them the procedures and the lesson materials. 

Then, I told the students who their cooperative team members would be to work together 
on the practice. Before starting on the 20 minutes practice activities, I reminded the students in 
both groups about the previously distributed guidelines for working effectively and efficiently in 
a team. Participants offered the academic incentive were also told that the group success was 
dependent on the success and contribution of every individual in the group and they were highly 
encouraged to work cooperatively. In order for each student in the team to receive the bonus 
points, each student in the group had to get 88% or higher on the post-test in order for all the 
individuals in the group to receive the bonus points. 

After working on the practice for 20 minutes, students were given the answers for the 
practice to see whether or not their answers were correct, and they were asked to discuss the 
answers with their team members for 5 minutes before returning the practice to the researchers. 
Then, the students took 10 minutes to answer the attitudinal survey individually, and returned 
their responses to the researchers. Before leaving the classroom, I reminded the cooperative 
learning groups who were offered the academic incentive about the importance of individual 
contribution in the post-test that would take place in the following session. But, the cooperative 
learning groups who were not offered an academic incentive were just told that in the coming 
session they would sit for post-test at the beginning of the session. 
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On the second day of the study, which was 5 days after the instruction and the practice 
for each group, the students completed individually a 20 minute post-test in the beginning of the 
session. Before taking the test, I reminded the cooperative groups who were offered the academic 
incentive about the importance of their contribution in order for their team members to receive 
the bonus points and about the test directions and time. At the end of the test, the researchers 
compiled the post-test responses from all the students. 

Results 

Concept Acquisition 

Concept acquisition was measured by a post-test that was administered at the end of the 
study. The test included items that measured recalling and applying the concepts for constructing 
objective test items that had been taught. Table 1 presents the means and standard deviations for 
each group on the concept acquisition skill post-test. A review of data revealed no violation of 
the assumptions of normality and homogeneity of variance. With alpha set to .05, and with 22 
subjects per treatment group, the probability of detecting a small difference between means was 
.96. 



The data were analyzed using analysis of variance (ANOVA). Results of ANOVA 
revealed that there was no significant main effect for academic incentive on test scores F (1,42) = 
2.59, p = .1 14. Students in the cooperative groups who were offered an academic incentive did 
not significantly score higher than the students in the cooperative groups who were not offered 
an academic incentive. These results did not support the hypothesis that the undergraduate 
students in the cooperative groups who were offered an academic incentive would do better on 



234 




recalling and applying concepts of constructing test items than undergraduate students in 
cooperative groups who were not offered an academic incentive. 

Learner’s Attitudes 

Learner’s attitudes towards instruction were measured by having students respond to an 
attitudinal survey that required them to report their degree of agreement on different statements 
concerning their level of attention, relevance, confidence, and satisfaction. Table 2 presents the 
means and standard deviations for each group on the attitudes. The data review for the 4 
categories of the attitudes survey revealed no violations for the assumptions of nonnality and 
homogeneity of variance. With alpha set at .05, and with 22 subjects per treatment group, the 
probability of detecting small difference between means for attention, relevance, confidence, and 
satisfaction was .85. 

The data were analyzed using multivariate analysis of variance (MANOVA) followed by 
one-way ANOVA. Results from MANOVA showed that the groups did not differ significantly 
on attitudes F(l,42) = 2.052. Students in cooperative groups who were offered an academic 
incentive did not have higher attention, relevance, confidence and satisfaction than students in 
cooperative groups who were not offered an academic incentive. Results from one-way ANOVA 
also indicated that the groups did not differ significantly on attention, relevance, and satisfaction 
F (1, 42) = 1.35, p = .251; F (1, 42) = .64, p = .427; and F (1, 42) = .62, p = .437 respectively. 
However, one-way ANOVA results revealed that there was a significant effect for the academic 
incentive on students’ confidence between the 2 groups F(l, 42) = 6.90, p < .05. Students in 
cooperative groups who were offered an academic incentive had higher confidence than students 
in cooperative groups who were not offered an academic incentive. The standardized difference 
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between these means was .81 standard deviations. According to Cohen (1997), this is generally 
considered to be a large effect. 

Given that there was no significant difference between groups on attention, relevance, 
and satisfaction; and a significant difference between groups on confidence, the attitudes results 
partially support the hypothesis that the undergraduate students in the cooperative groups who 
were offered an academic incentive would have better attitudes towards instruction than the 
undergraduate students in the cooperative groups who were not offered an academic incentive. 

Discussion 

The purpose of this study was to examine the effect of academic incentive use with 
cooperative learning groups on acquiring concepts of constructing objective test items, and on 
the attitudes towards instruction for undergraduate students in an introductory classroom 
assessment course. I provide a discussion of the results as they relate to other research conducted 
in the area. 

The first hypothesis that academic incentive use with cooperative learning groups would 
have a positive effect on students’ performance was not supported. The mean post-test 
performance for the students who were offered an academic incentive was not significantly 
higher than students who were not offered an academic incentive. As has been the case in 
previous studies (Brewer & Klein, 2006; Klein & Schnackenberg, 2000; Neihoff & Mesch, 
1990), the non significant effect of academic incentive on students’ perfonnance was most likely 
due to the well-designed instruction that was given to the participants. The effect of positive 
interdependence is likely to be weak when well-designed instructional materials are used 
(Brewer & Klein, 2006). 
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Additionally, why the academic incentive failed to have a positive effect on students’ 
performance can be explained from a variety of standpoints. First, the instructors confirmed that 
most of the students in this class were careless about the subject matter and were not focused on 
learning the various concepts for writing test items. The second explanation was most likely 
related to the absence of other elements that were crucial for the activity to be cooperative. One 
of these elements was social skills. The students did not have much experience working in teams 
in this class and they might not have the interpersonal and small group skills that allow them to 
contribute to the success of the cooperative effort (Johnson & Johnson, 1999). In other words, 
placing socially unskilled individuals in a group and telling them to cooperate does not guarantee 
that they will be able to do so effectively. 

The third explanation could be related to the high achievement goal (88%). According to 
Johnson, Johnson, and Smith (1998a), interdependence theory assumes that cooperative efforts 
are based on intrinsic motivation generated by interpersonal factors and the desire to achieve a 
significant goal. Since the students were rated by their instructors as being careless towards the 
class, they might have seen the set achievement grade as very high to achieve and they were not 
motivated by the academic incentive to attain the goal. In other words, as has been the case in 
previous research (Klein & Pridemore, 1992; Klein & Schnackenberg, 2000), the students might 
have not viewed the academic incentive as a motive that was supposed to strengthen positive 
interdependence or affiliation to the team. 

The second hypothesis that academic incentive would enhance students’ attitudes towards 
instruction was partially supported. Academic incentive enhanced students’ confidence level; 
however, it did not significantly affect the attention, relevance, and satisfaction of students 
between the two groups. Students who were offered the academic incentive felt that they were 
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confident that they teamed the subject matter. This finding partially corroborates previous 
research indicating that academic incentive increases the perception of positive interdependence 
and in return enhances attitudes (Johnson, Johnson, & Hollubec, 1994; Salvin, 1991; Webb, 
1997). In other words, the students who were offered the academic incentive felt confident that 
they could achieve the goal and receive the reward if they worked cooperatively. 

There are a few problems associated with this study. First, the majority of the students in 
the class were females, which limited the generalizability of results. Second, there were two 
different instructors who taught the different sections of the course. Even though they had the 
same teaching experience and they used the same teaching materials; they might have different 
teaching style. Third, the study took place before the students’ midterm exam in few days. They 
were preparing for the midterm exam and did not care for the study because it did not cover 
material that would be included in the exam. Finally, the relatively short duration of the 
treatment may have influenced results in this study. Extending the overall time for instruction 
may lead to results in favor of cooperative learning not found in this study. In other words, if the 
treatment was administered for a longer period of time in different sessions of the instruction, the 
academic incentive could have led to long-term effect and it would have been more possible to 
get favorable results. This study provides evidence that the students’ confidence can be enhanced 
by offering them an academic incentive when working cooperatively in groups. The study also 
supports previous studies that suggest that researchers who are comparing cooperative groups 
while using well structured material do not consistently find differences. Therefore, including 
higher-order problems on the achievement test could increase the benefits of this instructional 
strategy because aligning the posttest items with the practice activity and the objectives of the 
lesson did not lead to significant results. 
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In light of the growing prevalence of team work in education, future studies should 
continue to examine the use of incentives with cooperative groups. While findings from the 
current study do not support the assertion that academic incentives use with cooperative group 
affects students’ achievement, these claims should be tested with lessons that were not designed 
following an instructional systems approach. Researchers should also examine the effect of 
students’ characteristics and gender to discover which attributes influence achievement and 
attitudes in a cooperative setting. These suggestions will help us better understand the conditions 
under which cooperative learning is most effective, determine its appropriate use, and provide a 
richer picture of the benefits that cooperative learning may provide. 
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Table 1 



Means and Standard Deviations for Concept Acquisition Test 



Concept Acquisition Post-Test Scores 3 


Instructional Approach 


n 


M 


SD 


Academic Incentive 


22 


46.20 


13.87 


No Academic Incentive 


22 


40.20 


10.59 


Total 


44 


86.40 


24.46 



Note. d The maximum possible score was 100 points on the post-test. 
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Table 2 



Means and Standard Deviations for Attitudes 



Attitudes 3 






Attention 


Relevance 


Confidence 


Satisfaction 


Instructional 

Approach 


n 


M(SD) 


M(SD) 


M(SD) 


M(SD) 


Academic 

Incentive 


22 


3.40 (.49) 


3.70 (.60) 


3.98 (.40) 


2.80 (.87) 


No Academic 
Incentive 


22 


3.15 (.82) 


3.54 (.70) 


3.63 (.47) 


2.60 (.80) 


Total 


44 


6.55 (1.31) 


7.24(1.30) 


7.61 (.87) 


5.40(1.67) 



Note. a Measured on a five -point scale ranged from l(not true) to 5 (very true). 
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Abstract 

Due to the shift from the industrial age to information age, new educational needs require a new paradigm 
of education, one that is focused on learning rather than sorting students, one in which student progress is based on 
mastery, not on time. This new paradigm requires that technology serves new roles or functions. Reigeluth, Watson, 
Watson, Dutta, Chen & Powell (2008) identified four major functions and several secondary functions for 
technology to serve in the new paradigm. While the authors used the term Learning Management System for a 
system that serves all those functions, that term is often used to refer to course management systems and other 
systems that do not serve all those functions. To avoid confusion, we now call this system “PIES - Personalized 
Integrated Educational System”. This paper elaborates on the PIES functions identified by Reigeluth and his 
colleagues (2008) and addresses issues of system architecture and interfaces, including open educational resources 
and Web 2.0 tools. The paper describes recent advances in the design of a modern, information-age educational 
technology system that incorporates many vital functions to meet educational needs, including record keeping, 
planning, instmction, assessment, communications, administration, and more. It incorporates the latest Web 2.0 
tools and resources available. Having a flexible, open-source architecture, it has backward, current, and forward- 
looking capabilities to guide and assess students’ progress and help make available and deliver personalized 
instmction. 



Overview 

In recent years, a shift of the education paradigm from the industrial age to the information age has been 
discussed among educational researchers (Reigeluth & Duffy, 2008; Watson & Reigeluth, 2008). Reigeluth and 
Carr-Chellman (2009) proposed that a true paradigm shift in education could be realized by two developments: 
advanced technologies and the advancement of learner-centered psychological principles and methods of instmction. 
Technology plays a central role in the learner-centered paradigm (Reigeluth & Carr-Chellman, 2009). Reigeluth et 
al. (2008) identified four major functions and several secondary functions for technology to serve in the new 
paradigm. While the authors used the term Learning Management System for a system that serves all those 
functions, that term is often used to refer to course management systems and other systems that do not serve all 
those functions. To avoid confusion, we now call this system “PIES - Personalized Integrated Educational System”. 

Moreover, the term “Web 2.0” became popular when O’Reilly Media and MediaLive hosted the first Web 
2.0 conference in 2004. In his famous paper What is Web 2.0: Design Patterns and Business Models for the Next 
Generation of Software, O’Reilly elaborated on the term “Web 2.0” (O’Reilly, 2005). A wide variety of Web 2.0 
applications have been investigated by practitioners and researchers from different disciplines, including Wikis, 
Blogs, podcasts, simulations, serious games, virtual reality, mobile devices, social bookmarking, and social 
networking sites (Berlanga, Sloep, Van Rosmalen, Bitter-Rijpkema, & Koper, 2007; Boyd & Ellison, 2008; 

Farrelly, 2008). The rapid evolution of Web 2.0 Technologies has generated a new level of communication and 
interaction among learners. 

This paper elaborates on the PIES functions identified by Reigeluth et al. (2008) highlighted in Table 1 and 
addresses issues of system architecture and interfaces, including application and integration of open educational 
resources and Web 2.0 technologies. 
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Table 1. 



Four Major Functions & Secondary ; Functions of an Information-Age PIES identified by Reigelutli et al. (2008) 



Record-Keeping 


Planning 


Instruction 


Assessment 


Secondary 

Functions 


- Standards 


- Long Term Goals 


- Project Initiation 


- Presenting 


- Communication 


Inventory 






Authentic 






- Current Options 


- Instruction 


Tasks 


- General Student 


- Personal 


& 






Data 


Attainments 


Requirements 


- Project Support 


- Evaluating 




Inventory 






Student 


- School Personnel 




- Short-Term Goals 


- Instructional 


Performances 


Information 


- Personal 
Characteristic 


- Projects 


Development 


- Providing 


-PIES 


Inventory 


- Teams 




Immediate 

Feedback 


Administration 




- Roles 




- Certification of 










Attainments 






- Contracts 




- Developing 
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Assessment 










- Improving 
Instruction 
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Major Functions of PIES 

The four major functions of PIES include (I) record keeping, (2) planning, (3) instruction and (4) 
assessment for student learning. 

1. Record keeping: In the new paradigm of education, one of the most important functions is to keep a record of 
what each student has mastered. Current report cards only serve as tools for sorting the students, and they do not 
indicate what a student has actually achieved. The record keeping function has a number of sub-functions: (1) a 
standards inventory, which identifies and maps all the required standards and many optional standards in all 
domains, (2) a personal attainments inventory, which portrays the standards each student has mastered, and (3) a 
personal characteristics inventory’, which identifies each student's learning styles, profile of multiple intelligences, 
interests, major life events, and anything else that is useful in improving the learning experience for a student. 

2. Planning: Based on the data in the record keeping function, decisions must be made to plan for student learning. 
The planning function of PIES has many sub-functions to (1) help students, parents and teachers to set long term 
goals; (2) figure out current learning options that will help students to achieve their long term goals; (3) pick some 
of the current options and set them as short-term goals; (4) identify projects for achieving the short-term goals; (5) 
set project teams based on students' interests; (6) decide on the roles that the teacher, parents and others will play to 
support the student's learning in the projects, and (7) develop a contract that identifies goals, projects, teams, roles, 
deadlines, and milestones. 

3. Instruction: The instruction function of PIES helps students and teachers to (1) initiate each project, (2) provide 
resources/tools/guidance for instruction, (3) provide project support by monitoring and managing the project teams, 
and (4) guide teachers for developing new instruction, if needed. These sub-functions provide customized and 
learner centered instruction to maximize student learning. 



246 







4. Assessment: In the new paradigm teaching and testing are fully integrated. The sub-functions help teachers to 
(1) present authentic tasks for student assessment; (2) evaluate student performances based on those tasks, (3) 
generate immediate feedback on student performances, (4) provide certification when an attainment has been met, 
(5) develop student assessments such as rubrics, tests, etc., and (6) improve instruction and assessments based on 
their effectiveness. 



Secondary Functions of PIES 

There are also some secondary functions that are not directly related with student learning but can play a 
great role in facilitating student learning. (1) The communication function involves teacher communication and 
collaboration with other teachers, with students' parents, and with students. It also allows students to communicate 
and collaborate with each other to facilitate their learning. As mentioned at the beginning of this paper, Web 2.0 
technologies integrated in PIES will help generate a new level of communication and interaction among students, 
teachers, and parents. (2) The general student data function provides access to such data as the student’s name, 
address, birth date, parent information, health information, attendance, student’s mentor and other teachers, records 
of major life events, the school or learning community the student belongs to, the student’s home room, and 
community organizations with which the student is involved. With this function, PIES will gather, secure and allow 
convenient management of data such as those described above in order to support the learner-centered learning 
environment. (3) The school personnel information function provides access to such information as a staff member's 
name and address, assigned students, certifications and awards received, professional development plan and 
progress, and the teacher’s physical location. Appropriate management of the school personnel information will 
support the role of teachers as facilitators, coaches, and mentors that is required in a learner-centered environment 
(McCombs & Whisler, 1997). (4) The PIES administration function oversees the whole system, including restricted 
access to sensitive information about students. 

System Architecture and Interfaces 

The PIES concept is envisioned to be a comprehensive, fully-featured, open-source software application 
that is free for anyone to develop, modify, and redistribute. Many of the functions and features mentioned could be 
developed into an actual working PIES application using and combining smaller, separate and currently available 
open source applications. Some of these applications might include Linux (base operating system), MySQL 
(database), Dmpal (content management), Moodle (base LMS - if it can be extended), and Firefox or Flock (web 
browser). PIES would also include smaller built-in applications for chat, playing and authoring media, collaboration 
and social networking, and data integration and analysis. 

In terms of instructional content, PIES would also provide easy portal access to the growing number of 
open educational resource (OER) repositories such as Curriki, Connexions, TeacherTube and MERLOT. OERs are 
free educational resources that have been developed by a growing community of teachers, authors, and content 
producers, and are also open to customization and continual improvement for individualized student instruction. 

This cost-effective open curriculum of free resources would be owned by public taxpayer dollars and allow school 
funds to be released from textbooks and other commercial products and put toward more worthy education expenses 
like teacher salary and other more urgent expenses that better serve student needs (Wiley, 2009). The quality of this 
OER content may be easily peer-reviewed and rated by users. 

PIES would offer a flexible, customizable, easy-to-use, homepage interface such as in Elgg, Facebook, 
Ning, and iGoogle that would connect all PIES functions. The layout, basic functions, and preferences (RSS, text 
messages, chat, email, built-in media player, discussion boards, records of student attainments, teacher and course 
evaluations) would be highly customizable. The interface provides teachers and students with progress data, time on 
task, and many other metrics, while providing for social interaction and the “network effect” (Bush, 2009) of 
powerful mass-participation and collaboration. PIES would be ported and available to any internet-enabled device 
(PC, laptop, cell phone, PDA, etc.). Content could be downloaded, stored and made available “offline” for use 
without Internet access. All of these components of PIES are illustrated in Figures 1, 2, 3 and 4 below. 
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Figure 1. PIES- General Components 




Figure 2. PIES - Hardware Technology Components 
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Figure 3. PIES - Information- Age Functions 
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Figure 4. PIES - Open Source Architecture 



Future Activities 

Based on the PIES conceptual framework and design architecture, additional work needs to be done to 
further design and develop PIES. To this end, a group of researchers in our research group is in the process of 
finding funding to design and develop a true PIES, which will incorporate information-age functions and open 
educational resources in one integrated platform. 
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Purpose 

The purpose of this report is to describe the impact of a technology diffusion initiative at a private university in a 
large urban area. The university is a teaching university, rather than a research university, thus there is a strong 
focus on personal interactions with members of the community. The goal of the technology diffusion initiative 
described here is to infuse technology into the environment in an effort to aid in preparing students to develop the 
technical skills necessary to be successful in the 21 st century. 

The researchers have begun implementing technology diffusion practices that build off the social nature of the 
community in order to diffuse technology into the environment. Our technology diffusion initiative is inline with 
Bronfenbrenner’s (2005) bioecological model of human development that views human development within a 
complex system of relationships affected by multiple levels of the surrounding environment. With the use of Web 
2.0 technologies becoming more mainstream, the researchers see promise in their technology diffusion initiative, as 
we are able to skillfully match the social nature of the faculty and students within the university environment with 
contemporary technology tools and are able to successfully enhance technology diffusion in a way that complements 
the university environment. 



Background 

The need for faculty development is consistently identified as a primary factor influencing the adoption of new 
instructional technologies in higher education. Although needs have been assessed, and workshops are designed to 
meet these needs, workshops are not very well attended. Additionally, few faculty implement what they have 
learned at these workshops in their own classroom. The extant literature suggests that faculty are resistant to 
attending workshops because: 

1 . Programs are developed without any knowledge of or consideration for how faculty learn and grow 
(Chism, 2004); and 

2. Psychosocial factors influence the willingness of faculty to attend group workshops. (Cravener, 1999). 

The observations regarding the resistance of faculty to attending workshops described abovecould have a negative 
impact in preparing students to develop the technological skills necessary to be successful in the 21st Century. 

The researchers employ Zhao, Lei, and Frank’s “The Social Life of Technology: An Ecological Analysis of 
Technology Diffusion in Schools” (2006) as the initial framework for their initiative. Zhao et al’s (2006) framework 
builds off of Bronfenbrenner’s model of human development and applies the model to technology diffusion. As 
Zhao et al’s framework takes into consideration the dynamic nature of the environment in which technology is being 
adopted, and the context into which the technology is being diffused. 

Additionally, the researchers provide two different perspectives for their initiatives. One perspective is that of a 
faculty member in the School of Education, which addresses instructional practices that promote technology 
diffusion. The other perspective is that of a senior instructional designer at the same university, which addresses 
faculty development practices that promote technology diffusion. For each type of initiative, instructional practices 
are discussed first followed by professional development. Data is gathered at different points during the initiative in 
order to determine the effectiveness of the different elements of our initiative. Data is gathered in the form of end- 
of-course evaluations, professional development surveys, and personal correspondence. 
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The Initiative 



Focus on Building Networks of Teachers. 

Instructional Practices 

• Use Web 2.0 technologies. The researchers have begun using Web 2.0 technologies in some of their 
classes so that part of the learning takes place outside of course management systems. The purpose of this 
practice is to foster the building of a network of teachers inside and outside of the classroom. By having 
some of the instructional activities take place outside of class, students are enabled to continue the learning 
and collaboration long after the class is complete. In teacher education courses this enables a network of 
teachers to evolve and grow. 

Based on the feedback from students who have used Web 2.0 tools in their online courses, we have found 
that many students continue to use the Web 2.0 technologies outside of class. In some cases the students 
use the technology for other courses. For instance, students continue to use social bookmarking in other 
courses as they continue to share resources with others well after the online class is complete. 

Furthermore, other students use Web 2.0 technologies in their professional lives. For example students are 
using blogs and social bookmarking to communicate and share with parents and other teachers’ in their 
professional environment. These are just two examples of the important ways that our technology diffusion 
initiative has impacted students’ use of technology long after class work is complete. 

Faculty Development 

• Attend Departmental Faculty Meetings. Traditionally, instructional designers do not attend 
departmental faculty meetings. Rather Instructional Designers invite faculty to attend workshops and 
training sessions based on perceived need or faculty responses to surveys. This is similar to networking in 
business, in which connecting one person with another person for the right reason is a powerful networking 
tool (Levin, 2003). In an analogous manner, the ability to create connections between a person’s 
professional development needs with the appropriate professional development resource is critical to 
technology diffusion in education. By attending faculty meetings, the senior instructional designer can 
better understand the needs of faculty by embedding themselves in the faculty discussions and connecting 
faculty with the appropriate professional development tools. 

The faculty feedback is positive because the designer was able to tailor their professional development to 
very unique subject-specific needs. The senior designer has made a concerted effort to attend faculty 
meetings with the purpose of listening to faculty needs, and following through by designing custom 
training to meet faculty needs. 

Give the Idea Some Time and Encourage Play 
Instructional Practices 

• Pre-Instruction for Online Classes. One instructional method that the researchers are beginning to 
implement is the use of pre-instruction for some classes. Prior to the beginning of a class, students are 
asked to explore specific Web 2.0 technologies that they will use in class. This affords the student some 
time to explore the technology without the deadlines, and without the other stressors typically associated 
with using the new tools for instructional purposes. Additionally, pre-instruction gives the student time to 
fully understand how to use the tool. 

For instance, prior to an online technology class, students are asked to create a blog and a social 
bookmarking site. The goal is that by the time the class begins, the students will be familiar with the blog 
and social bookmarking tools so that they can then focus on the instruction rather than how to post a 
message or create a bookmark. 

Based on feedback from students who participated in the pre-instruction, students found the pre-instruction 
to be very helpful and students encourage the continuation of such practices. Suggestions based on the 
feedback is to provide instruction for specific tasks are similar to the tasks that they will be conducting 
online. This allows students the time to practice specific tasks with the technology prior to the start of the 
course. 
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Faculty Development 

• Follow-ups. The instructional designer contacts workshop participants several days after a workshop 
to further encourage the use of a technology for instruction, and to answer follow-up questions. This gives 
workshop attendees time to think about what they have learned in the workshop, and allows them time to 
further explore the technology. The follow-ups help to assure that the workshop participants are able to 
apply what they learned at the workshop, and provide the extra support necessary for implementation. This 
also allows workshop participants to see that the workshop facilitator wants to put action into their words 
(Levin, 2003). 

Feedback indicated that faculty appreciated the reminders, and the additional support, to help them initiate 
their instructional efforts subsequent to the workshop. 

• Taste of Technology’. The instructional designer provides 'Tastes of Technology’ seminars where they 
take 15 minutes to discuss a new technology and showcase its use. The attendees have the opportunity to 
discuss other possible uses of the showcased technology. Based upon the discussion generated from the 
seminar, further workshops or training are designed if there is sufficient interest. 

Feedback regarding the Tastes of Technology indicated that the information was immediately applicable. 
Feedback also indicated an appreciation for the abbreviated workshop. The faculty were able to learn about 
the technology in a short, concentrated seminar. Feedback indicated that it was useful information by 
didn’t require a major commitment of time. 

• Hybrid Professional Development. Offer a Web 2.0 Tools for Teaching, Learning, and Research 
hybrid course. The success of hybrid courses in formal education environments has spurred the motivation 
for this professional development initiative (Bernard, R. M., Abrami, P. C., Lou, Y., Borkokhovski, E., 
Wade, A., Wozney, L., Wallet, P. A., Fiset, M., and Fluang, B. 2004; Rovai and Jordan, 2005; U.S. 
Department of Education, 2009). Begin the professional development in a face-to-face environment and 
then allow participants time to interact and learn from the content online. 

Faculty appreciated the flexibility offered by this type of professional development, especially being able to 
work at their own pace online and the ability to ask questions during the face-to-face meetings. The hybrid 
professional development participants did express the desire for the course to be longer, so that they had 
more time between sessions. At the time of this writing, the hybrid professional development courses are 
four weeks in duration. 



Connect to Existing Practices, Find the Right Niche, and Treat Computers as Part of the Existing Pedagogical 
Ecology. 

Instructional Practices 

• Create Dynamic Instruction. In order to foster the adoption of technology for instruction, it is 
important to make sure that the given technology actually meets the needs of those in the instructional 
environment. People will not adopt a technology if they can not use it. Thus, the researchers have 
incorporated into some of their courses, dynamic projects that are customizable to the professional needs of 
the students. 

The use of customizable projects was greatly received by students. Feedback indicated that many of the 
students utilized the projects in their professional situation as soon as their professional situation allowed. 
Additionally, feedback from students indicated that they plan on making additional projects for other 
aspects of their professional situation based on the course projects. This indicates that once they were able 
to create a successful and usable project for their own professional situation, the students see other venues 
in which they are able to incorporate other projects/technology into their professional situation. An 
outcome that often times one can only hope for in a best case scenario, and which we realize here. 

• Use the Language of the Students. The researchers have implemented the use of social bookmarking 
tools that utilize tagging. This way students and teachers can share resources using terms that are not 
"techno jargon" but rather the terms are familiar to the students and relevant to their teaching. The use of a 
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social bookmarking tools that utilize tagging is one of the great examples of how technology can be 
diffused into the environment by using the language of the community. 

Feedback from students suggested that tagging is a difficult concept to understand initially. However, once 
students become more familiar with the concept, and once they see the benefits, they are able to utilize the 
tag feature more effectively. Extensive instruction on creating tags is necessary for this type of 
instructional practice to be effective. The extra training is warranted given the positive outcome. 

• Focus on the Instruction not the Technology’. The researchers have designed some of their courses to 
focus on how the technology can be used in the classroom rather than focusing on the technology itself. 

While every effort was made to focus on the instructional use of the technology, feedback indicated that 
students focus on the technology itself, as compared to the use of the technology, perhaps one of the 
biggest hurdles with all technology application, inside the classroom or out. 

Faculty Development 

• Partnerships. Partner with faculty to teach their students the applications of technologies that are used 
in their classes for completing activities and projects. By becoming a resource for teachers and responding 
to requests from faculty, instructional designers become valuable resources in their environment (Levin, 
2003). Thus, instructional designers are able to lead the way with technology diffusion. 

Faculty appreciated making their own projects at their own pace that they could immediately apply to their 
instructional environments. 

• Focus on Faculty Needs Not the Technology’. Faculty development initiatives are designed based on 
faculty needs not technology tools. Design and delivery of opportunities customized to faculty needs by 
department, and individual follow up with each participant after group training has been delivered. 

Feedback expressed by the faculty indicated that faculty liked the customized training based on the faculty 
meetings and their unique instructional needs. They indicated that they were able to apply immediately 
what they learned. 



Conclusion 

By blending traditional social networking with modern technologies that enable social networking the researchers 
intend to increase the use of technology for instruction at their university. The feedback that has been received 
indicates that forward progress is being made with the technology diffusion initiatives. In regards to teacher 
education course, our technology diffusion initiative has had important impacts in the professional lives of our 
students long after class work is complete. Similar results have been found with the faculty development aspect of 
this report as faculty are using technology in their classes long after the professional development initiatives are 
over. 
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Abstract 

This paper examines the use of commercial-off-the-shelf (COTS) games in formal educational settings. A 
survey method was employed to determine which COTS video games are appropriated for formal educational 
settings, and the benefits and barriers to integration in formal educational settings. Ninety-nine members of the 
Special Interest Group of Games and Simulations (SIGGS) in the International Society for Technology in Education 
(ISTE) responded to the survey. The results suggest that greatest benefits to integrating COTS videos games include 
developing cognitive skills, teaching complex problem-solving, accepting and learning from mistakes, and learning 
by doing. The greatest barriers include teachers’ background in games and technology, perception of games, lack of 
financial resources, alignment with the curriculum, lack of evidence to support their use, and lack of time to 
integrate into the classroom. Conclusions are provided. 

Keywords: Commercial-off-the-shelf, formal educational settings, benefits, barriers. 

Introduction 

In recent years, attention has been placed on the utility of commercial-off-the-shelf (COTS) video games in 
formal and informal educational settings (Becker, & Jocaobsen, 2005; Becker, 2006; Sanford, Ulicsak, Facer, & 
Rudd, 2006; Ertzberger, 2008; Charsky & Mims, 2008), particularly for military training applications (Barlow, 
Morrison, & Easton, 2002; Morrison & Barlow, 2004; Fong, 2006). COTS video games can be purchased from 
brick-and-mortar businesses, online, and via catalogs. COTS video games, unlike serious games (Michael & Chen, 
2005), are not designed for the sole purpose of training, educating or informing, but rather for the primary goal of 
entertainment. Some COTS video games are deemed appropriate for formal educational settings (Sanford, Ulicsak, 
Facer, & Rudd, 2006). Charsky and Mims (2008) note “creating game -based learning environments or experiences 
using COTS games is becoming an increasingly tenable, valuable, and popular instructional strategy” (p. 38). COTS 
video games for formal educational settings should exhibit broad appropriateness for school settings, engage players 
with authentic content and challenges, and show the presence of clear causal relationships between game variables 
(Sanford, Ulicsak, Facer, & Rudd, 2006). 

However, as discussed by Becker (2006), there remains resistance to the use of COTS games in formal 
educational settings. Squire (2003) suggests that integrating games into formal educational settings potentially raises 
as many problems as it solves. COTS video games may not appeal to every student equally (Ke, 2008) and students 
may be distracted by game -playing, and consequently, not achieve the educational goals (Miller, Lehman, & 
Koedinger, 1999). Egenfeldt -Nielsen (2004) states there are numerous barriers to successfully integrating games; 
including scheduling, setting, class expectations, teacher background, genre knowledge, technical problems, teacher 
preparation, perception of games, class size, and priority issues - all of which place a tremendous burden on 
teachers. Thus, the use of COTS video games in formal educational setting remains a topic of debate among 
researchers and educators alike. 



Commercial-Off-The-Shelf Games in Education 

Becker and Jacobsen (2005) surveyed 109 educators about the utility of computer and video games in a 
classroom setting. Fifty-seven percent of their respondents had used computer or video games in their classroom 
practice. More than half of the educators indicated that the teachers themselves, either through their own efforts, 
through professional development opportunities, or through help from their students or nearby colleagues provided 
the greatest amount of help in integrating computer or video games into their classroom practice. When asked about 
barriers, more than 70% of the respondents indicated that there was not enough time to integrate games, not enough 
classroom computers, not enough games within the schools, or that integrating games was not a school priority. 
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Ertzberger (2008) surveyed 390 educators from 1 10 different schools about the utility of video games as 
instructional tools. His results suggest the most important factors in promoting the use of video games are the games 
alignment to curriculum, reliability of the technology, and hands on training available. Results indicate the biggest 
deterrents to the use of video games were the expense of video games, lack of time to create video games, lack of 
relevance to curriculum, and lack of the needed technology. Participants boasted the ability to individualize 
instruction, and increase motivation as benefits to video game use. 

Future Lab, a non-profit organization for innovation in teaching, conducted a survey of primary and 
secondary teachers’ (N = 1,000) attitudes towards games in education (Future Lab, 2005). Their survey was not 
specifically about COTS video games. Their findings suggest that 72% of the teachers do not play video games in 
their leisure time and 69% have not used games in their classrooms for educational purposes. Though 59% of the 
teachers would consider using games in their classroom, 49% of same population see the largest integration barrier 
as access to equipment (Future Lab, 2005). 

Kerbitchi (2009) examined factors affecting teachers’ adoption of computer games using a case study 
method. The results were compared with the existing literature on the adoption of educational software, and the 
barriers in the use of educational computer games in formal K-12 settings. The findings showed that adoption 
attributes for the games and other educational software had a similar pattern from high to low significance on 
relative advantage, compatibility, complexity, trialability and observability (Rogers, 2003) and that the game 
adoption factors were more inclusive than the barriers of using the computer games. 

While these studies have certainly informed educational research, they leave much to be desired. None one 
of the aforementioned studies specifically examined the role of COTS video games in formal educational settings. 
Further, in another study, the results confirmed that the majority of the educators were not gamers (72%) and had 
not integrated games into their classrooms (69%). These findings are alarming and confirm there is a dearth of 
existing research examining role of COTS video games in formal educational settings. 

Research Questions 

Therefore, this research sought to address the following research questions: 

1 . What commercial-off-the-shelf video games do professionals deem appropriate for formal educational 
settings? 

2. What are the benefits of integrating commercial-off-the-shelf games in formal educational settings? 

3. What are the barriers to integrating commercial-off-the-shelf games in formal educational settings? 

Method 



Participants 

Participants included members of the Special Interest Group of Games and Simulations (SIGGS) in the 
International Society for Technology in Education (ISTE). One-hundred fifty-four participants opened the survey, 
and of those participants, A=137 provided a response to at least one item. During this period, the SIGGS had 
approximately 1,400 members, which is a response rate of approximately 10%. Those participants that did not 
complete at least 75% of the survey items were removed from the analysis resulting in a final set of n= 99 
participants. 

As can be gleaned in Table 1, approximately 68% of the participants were female. The majority of the 
participants are in the age range of 36 to 65 years old. More than 80% of the participants reported salary ranges from 
$30,000 to $100,000 per year. More than 80% of the participants reported a White/Caucasian ethnicity. One 
hundred percent of the sample held at least a bachelors degree with approximately 49% holding a masters degree. 



Table 1. Demographic characteristics of sample. 

Category n % 

Gender 



Female 


67 


67.68 


Male 


31 


31.31 


Age Range 
0-25 


3 


3.03 


26-35 


15 


15.15 


36-45 


27 


27.27 


46-55 


27 


27.27 
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Category 


n 


% 


56-65 


24 


24.24 


>65 


2 


2.02 


Salary Range 


N/A 


6 


6.06 


0 - $30;000 


1 


1.01 


$30;001 - $50;000 


18 


18.18 


$50;001 - $75;000 


37 


37.37 


$75;001 - $100;000 


26 


26.26 


$ 1 00;00 1 - $150;000 


7 


7.07 


> $150;000 


2 


2.02 


Ethnicity 


Asian 


2 


2.02 


Black/African American 


3 


3.03 


Hawaiian/Pacific Islander 


2 


2.02 


Hispanic/Latino 


2 


2.02 


White/Caucasian 


84 


84.85 


Other 


6 


6.06 


Highest Degree Earned 


Bachelors Degree 


23 


23.23 


Masters Degree 


49 


49.49 


Specialist Degree 


5 


5.05 


Doctorate Degree 


22 


22.22 



Ninety-six percent of the participants were from the United States with the remaining from Canada. 
Approximately 68% of the participants reported playing computer games and 51% reported playing video games. 
The job titles of the participants represented a wide range of careers from teachers to college professors to 
educational technologists. Participants reported an average of 15.36 years ( SD = 9.35) in educational technology- 
related professions. 

Instrument 

The instmment was divided into four sections: background information, COTS, COTS barriers, and COTS 
benefits. The background information section included items pertaining to age, gender, ethnicity, salary range, 
educational attainment, game play information, and job title information. The COTS section included 19 popular 
COTS games for multiple platforms derived from multiple sources (Kirriemuir, 2005; Prensky, 2010) and with a 
response scale of Very Appropriate, Appropriate, Neutral, Inappropriate, and Very Inappropriate. 

The COTS games barriers section included 23 potential barriers to the integration of COTS games in 
formal educational settings with a response scale of Very Significant Barrier, Significant Barrier, Neutral, 
Insignificant Barrier, and Very Insignificant Barrier. Finally, the COTS games benefits section included 18 potential 
benefits of integrating COTS games into a formal educational setting with a response scale of Very Significant 
Benefit, Significant Benefit, Neutral, Insignificant Benefit, and Very Insignificant Benefit. Both the COTS games 
benefits and barriers scales were derived from previous research (Becker, & Jocaobsen, 2005; Ertzberger, 2008). All 
scales also included a Do not Know option. 

Procedures 

The survey was made accessible in a web-based format using SelectSurvey. The survey was available 
during a three-week period, and during this time, two reminder emails were sent to members of the SIGGS. 
Participants were informed the purpose of this project was to examine characteristics related to the use of COTS 
games in formal educational settings. 

Data Analysis 

This study employed a mixed-method approach (Tashakkori & Teddlie, 1998) involving both quantitative 
and qualitative procedures used concurrently and independently. Quantitative analyses of the data included 
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descriptive analysis of response frequencies and measures of variation and central tendency (Knupfer & McLellan, 
1996), and internal consistency reliability analysis (Cronbach’s alpha). Three open-ended survey items were 
included in the instrument for the purpose of gaining further insight into COTS games, benefits and barriers. The 
data were analyzed using a constant comparative method (Glaser, 1965; Glaser, 1967). 

Results 

The results of this research must be viewed in light of its limitations. The sample had a low response rate. 
Prior research suggests that average online response rates fall somewhere within the range of 24% - 39% (Cook et 
ah, 2000, Sheehan, 2001). Our research only achieved 10% response rate. This low response rate may not accurately 
represent the perceptions of the population. Additionally, the survey itself is a limitation. The questions were few 
and the content of a limited nature. Further research is necessary to validate an instalment for wide scale use. In 
light of these limitations, this research has resulted in several interesting findings. 

Commercial-off-the-shelf (COTS) Video Games 

The internal consistency reliability for the scale was high at a = .89. As can be seen in Table 2, many of the 
participants were not familiar with a handful of the COTS games listed in the survey. Big Brain Academy, Brain 
Age, and Oregon Trail had item averages at or above 4.0, indicating that those that were aware of the games, 
deemed them appropriate for a formal educational setting. Other games, however, had high averages (> 3.5), 
including Civilization III, DimensionM, Railroad Tycoon, Rise of Nations, The Sims, Wii Sports, and Zoo Tycoon. 
The only game clearly identified as not appropriate was the Grand Theft Auto series with an item average of 1.83 
OSD =1.22). 



Table 2. COTS games scale with descriptive statistics. 



COTS Game 


n 


DK 


VI 


I 


N 


A 


VA 


M* 


SD* 


Big Brain Academy 


95 


64.65 


1.01 


0.00 


6.06 


11.11 


13.13 


4.13 


0.96 


Brain Age 


95 


60.61 


2.02 


0.00 


5.05 


15.15 


13.13 


4.06 


1.03 


Civilization III 


93 


42.42 


3.03 


1.01 


8.08 


29.29 


10.10 


3.82 


0.97 


Dimension M 


92 


69.70 


3.03 


1.01 


4.04 


9.09 


6.06 


3.61 


1.31 


Endless Ocean 


92 


60.61 


9.09 


1.01 


4.04 


12.12 


6.06 


3.16 


1.53 


Grand Theft Auto 


91 


20.20 


42.42 


13.13 


5.05 


8.08 


3.03 


1.83 


1.22 


Guitar Hero 


91 


19.19 


7.07 


15.15 


19.19 


24.24 


7.07 


3.13 


1.15 


Half-Life 


91 


47.47 


6.06 


5.05 


12.12 


9.09 


12.12 


3.36 


1.37 


Oregon Trail 


89 


18.18 


1.01 


1.01 


5.05 


30.30 


34.34 


4.34 


0.79 


Railroad Tycoon 


89 


39.39 


1.01 


1.01 


8.08 


28.28 


12.12 


3.98 


0.82 


Rise of Nations 


89 


42.42 


1.01 


3.03 


5.05 


26.26 


12.12 


3.96 


0.91 


Simcity 


89 


27.27 


3.03 


4.04 


13.13 


25.25 


17.17 


3.79 


1.07 


Super Mario Galaxy 


89 


39.39 


3.03 


16.16 


20.20 


9.09 


2.02 


2.82 


0.94 


The Sims 


89 


34.34 


2.02 


6.06 


18.18 


20.20 


9.09 


3.51 


1.02 


The Legend of Zelda 


89 


51.52 


5.05 


8.08 


16.16 


6.06 


3.03 


2.84 


1.10 


Trauma Center: Under the Knife 


88 


52.53 


6.06 


3.03 


9.09 


13.13 


5.05 


3.22 


1.29 


Wii Sports 


86 


21.21 


1.01 


4.04 


16.16 


30.30 


14.14 


3.80 


0.90 


World of Warcraft 


80 


41.41 


9.09 


10.10 


13.13 


5.05 


2.02 


2.51 


1.14 


Zoo Tycoon 


64 


31.31 


1.01 


1.01 


8.08 


13.13 


10.10 


3.91 


0.98 



n=Number of respondents, DK = Do not Know, VI = Very’ Inappropriate, I = Inappropriate, N = Neutral, A = 
Appropriate, VA = Very Appropriate, M = Item average, SD = Item Standard deviation, *Note the average and 
standard deviations do not include the Do not Know response. 



Participants also had the option of listing COTS games that were not listed in the scale. These responses are 
summarized in Table 3. Only those with two or more responses are shown. In total, 22 unique games were provided. 
Of the 99 participants, only 23 provided a response with an average of 5.13 words (SD = 4.74). Of those listed, both 
Spore and Risk occurred most. 
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Table 3. Other COTS games listed. 



COTS Game 


n 


Age of Empires 


2 


Spore 


3 


Risk 


3 


Math Blaster 


2 


Carmen San Diego 


2 


Making History 


2 



Benefits to COTS Video Games 

The internal consistency reliability for the scale was also high at a = .93. Table 4 shows the results. The 
highest rated item averages (> 4.0) for the support of COTS in formal educational settings are to develop cognitive 
skills (M= 4.31, SD = 0.64), develop spatial relation skills (M= 4.14, SD = 0.72), develop motor skills (M= 4.12, 
SD = 0.73), knowledge acquisition (M = 4.18, SD = 0.76), increase memory capacity (M= 4.02, SD = 0.85), 
teaching complex problem-solving (M= 4.45, SD = 0.66), increase creativity (M= 4.15, SD = 0.91), transfer 
knowledge to real-world situations (M= 4.17, SD = 0.77), accept and learn from mistakes (M= 4.28, SD = 0.75), 
learning by doing (M= 4.26, SD = 0.70), increase self-esteem and self-confidence (M= 4.00, SD = 0.88), 
incorporate technology student’s use every day (M = 4.18, SD = 0.83), and to promote differentiated instruction ( M 
= 4.17, SD = 0.81). Notably, none of the potential benefit listed are below the central point (3.0). 



Table 4. Benefits to COTS video games. 



Benefit 


n 


DK 


VIB 


IB 


N 


IB 


VIB 


M* 


SD* 


Eliminate the digital generation 
gap 


98 


4.04 


1.01 


6.06 


21.21 


45.45 


21.21 


3.84 


0.88 


Develop cognitive skills 


98 


2.02 


0.00 


1.01 


6.06 


51.52 


38.38 


4.31 


0.64 


Develop spatial relations skills 


98 


3.03 


0.00 


3.03 


10.10 


53.54 


29.29 


4.14 


0.72 


Develop motor skills 


98 


3.03 


0.00 


1.01 


11.11 


55.56 


27.27 


4.12 


0.73 


Knowledge acquisition 


98 


1.01 


0.00 


2.02 


15.15 


44.44 


36.36 


4.18 


0.76 


Increase memory capacity 


98 


5.05 


0.00 


4.04 


20.20 


39.39 


30.30 


4.02 


0.85 


Teach complex problem solving 


98 


0.00 


0.00 


1.01 


6.06 


39.39 


52.53 


4.45 


0.66 


Increase creativity 


98 


1.01 


1.01 


4.04 


15.15 


36.36 


41.41 


4.15 


0.91 


Transfer of knowledge to real- 
world situations 


98 


3.03 


0.00 


2.02 


17.17 


39.39 


37.37 


4.17 


0.79 


Accept and learn from mistakes 


98 


3.03 


0.00 


1.01 


14.14 


37.37 


43.43 


4.28 


0.75 


Learning by doing 


98 


3.03 


0.00 


0.00 


14.14 


42.42 


39.39 


4.26 


0.70 


Increase self-esteem and self- 
confidence 


98 


3.03 


1.01 


4.04 


18.18 


43.43 


29.29 


4.00 


0.88 


Promote social skills 


98 


5.05 


6.06 


8.08 


21.21 


37.37 


21.21 


3.63 


1.12 


Teach students how to role-play 


96 


3.03 


1.01 


5.05 


25.25 


45.45 


17.17 


3.77 


0.85 


Incorporating technology that 
student’s use everyday 


90 


1.01 


0.00 


4.04 


12.12 


37.37 


36.36 


4.18 


0.83 


COTS games used as part of a 
reward system (i.e. work 
completion, etc.) 


98 


5.05 


10.10 


4.04 


32.32 


34.34 


13.13 


3.39 


1.12 
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Benefit 


n 


DK 


VIB 


IB 


N 


IB 


VIB 


M* 


SD* 


Promotes healthy competition 


96 


5.05 


2.02 


7.07 


26.26 


41.41 


15.15 


3.66 


0.92 


Promotes differentiation of 
instruction 


96 


6.06 


1.01 


1.01 


14.14 


40.40 


34.34 


4.17 


0.81 



n=Number of respondents, DK = Do not Know, VIB = Very’ Insignificant Benefit, IB = Insignificant Benefit, N = 
Neutral, SB = Significant Benefit, VIB = Very Significant Benefit, M = Item average, SD = Item Standard deviation, 
*Note the average and standard deviations do not include the Do not Know response. 



Participants also had the option of listing potential benefits not listed in the scale. Eighteen participants 
provided free-form responses averaging 1 1.05 (£D=9.94) words in length. Results suggest, in addition to those 
listed, increased communication among students and teachers, increased teamwork and collaboration among 
students, stealth assessment, and students learning the design of games themselves are additional potential benefits. 

Barriers to COTS Video Games 

The Cronbach;s alpha was high at a = .92. Table 5 shows the results. The greatest barriers (4 >) to the 
integration of COTS games include teachers’ background in games and technology (M= 4.33; SD = 0.86), 
perception of games (M= 4.23; SD = 0.85), lack of financial resources (M= 4.13; SD = 1.01), alignment with the 
curriculum (M= 4.09; SD = 1.00), lack of evidence to support their use (M= 4.04; SD = 0.89), and lack of time to 
integrate into the classroom (M= 4.23; SD = 0.85). 



Table 5. Barriers to COTS video games. 



Barrier 


n 


DK 


VIB 


IB 


N 


IB 


VIB 


M 


SD 


Lack of parental consent 


99 


6.06 


3.03 


13.13 


27.27 


33.33 


17.17 


3.52 


1.05 


Parental perceptions about 
games 


99 


5.05 


4.04 


7.07 


17.17 


38.38 


28.28 


3.84 


1.07 


COTS game ratings 


99 


14.14 


1.01 


11.11 


29.29 


34.34 


10.10 


3.48 


0.91 


Lack of access to technology 


99 


2.02 


3.03 


11.11 


12.12 


30.30 


41.41 


3.98 


1.14 


Lack of administrative support 


99 


3.03 


3.03 


8.08 


15.15 


31.31 


39.39 


3.99 


1.09 


Teacher’s background in games 
and technology 


99 


1.01 


2.02 


2.02 


7.07 


38.38 


49.49 


4.33 


0.86 


Perception of games 


99 


2.02 


2.02 


2.02 


8.08 


45.45 


40.40 


4.23 


0.85 


Training regarding the new 
technology 


99 


2.02 


3.03 


7.07 


12.12 


48.48 


27.27 


3.92 


0.99 


Technology reliability 


99 


1.01 


4.04 


14.14 


24.24 


36.36 


20.20 


3.55 


1.09 


Lack of financial resources 


99 


2.02 


2.02 


6.06 


13.13 


32.32 


44.44 


4.13 


1.01 


Alignment of curriculum 


99 


3.03 


1.01 


7.07 


16.16 


30.30 


42.42 


4.09 


1.00 


Lack of evidence to support 
their use 


99 


2.02 


1.01 


6.06 


12.12 


47.47 


31.31 


4.04 


0.89 


Lack of technical support 


99 


1.01 


1.01 


10.10 


25.25 


40.40 


22.22 


3.73 


0.96 


Lack of time to familiarize 
oneself with the technology 


99 


2.02 


2.02 


5.05 


15.15 


48.48 


27.27 


3.96 


0.91 


Lack of time to integrate into 
classroom 


99 


3.03 


0.00 


7.07 


15.15 


43.43 


31.31 


4.02 


0.88 


Class expectations 


99 


3.03 


2.02 


8.08 


27.27 


44.44 


15.15 


3.65 


0.92 


Class size 


99 


3.03 


6.06 


11.11 


24.24 


39.39 


16.16 


3.50 


1.10 


Lack of student motivation 


99 


2.02 


21.21 


31.31 


25.25 


15.15 


5.05 


2.51 


1.15 


Lack of teacher motivation 


99 


3.03 


3.03 


10.10 


14.14 


40.40 


29.29 


3.85 


1.07 


Lack of access to professional 
development 


99 


3.03 


3.03 


7.07 


18.18 


46.46 


22.22 


3.80 


0.98 


Lack of access to technology 
outside of schools 


98 


3.03 


4.04 


15.15 


28.28 


28.28 


20.20 


3.47 


1.12 
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Barrier 


n 


DK 


VIB 


IB 


N 


IB 


VIB 


M 


SD 


Conflict between teacher’s 
interests and teacher’s 
responsibilities 


87 


5.05 


0.00 


10.10 


16.16 


40.40 


16.16 


3.76 


0.91 


Concern for how students will 
be affected by the integration of 
games into the classroom 


87 


4.04 


2.02 


17.17 


25.25 


30.30 


9.09 


3.33 


1.00 



n=Number of respondents, DK = Do not Know, VIB = Very’ Insignificant Barrier, IB = Insignificant Barrier, N = 
Neutral, SB = Significant Barrier, VIB = Very’ Significant Barrier, M = Item average, SD = Item Standard 
deviation, *Note the average and standard deviations do not include the Do not Know response. 



Participants also had the option of listing potential barriers not listed in the scale. Twelve participants 
provided responses with an average length of 16.08 (SD=19.28) words. Additional barriers to integrating COTS 
video games include gender issues, equity issues for students with special needs (e.g., visually impaired), lack of 
games aligned with standardized tests, unhealthy competition, and Internet safety. 

Discussion 

One respondent captured the essence of perhaps the most significant barrier to using COTS video games in 
formal settings by stating “I must teach to these academic standards; my principal will give me a bad evaluation if I 
am not teaching the standards and we are playing games instead” (Respondent, 2009). This statement reiterates the 
ongoing challenge educators face in attempting to pioneer innovative instructional approaches employing 
technology. The perception of COTS video games is itself a major barrier as confirmed by our findings. Educators 
must make the case to leadership of the value of integrating COTS video games in the classroom. This inevitably 
requires more evidence to substantiate the use of COTS video games, which was identified as another significant 
barrier. We believe these findings, at minimum, make a call to educators and researchers alike to spearhead this 
complicated domain. 

The results of our research corroborate the findings of prior research. For instance, lack of time to integrate 
into the curriculum was perceived as a major barrier in Becker and Jocaobsen findings and in our own results. 
Additionally, Ertzberger (2008) found that lack of alignment to the curriculum and lack of financial resources were 
also significant barriers. Our findings provide further evidence of these barriers in integrating video games in formal 
educational settings. In addition, our research has identified a teachers’ background in games and technology, 
perception of games in general, and lack of evidence to support their use as significant barriers. 

In exploring potential benefits, the greatest benefits identified include developing cognitive skills, teaching 
complex problem-solving, accepting and learning from mistakes, and learning by doing. Additionally, qualitative 
findings suggested increased communication among students and teachers, increased teamwork and collaboration 
among students, stealth assessment, and students learning the design of games themselves are additional potential 
benefits. These findings are analogous to prior research suggesting video games are a tool to “use action instead of 
explanation, create personal motivation and satisfaction, provide interactive decision-making context” (Kebritchi, 
2008, p. 15), and can promote collaboration among learners (Kaptelin & Cole, 2002). While this research has 
explored the many perceived benefits of COTS video games, further research is necessary to examine the efficacy of 
these benefits in experimental and quasi-experimental settings. Put simply, more research is necessary. 

Our research has also identified COTS video games that are deemed appropriate for formal educational 
settings. These games include Big Brain Academy, Brain Age, Oregon Trail Civilization III, DimensionM, Railroad 
Tycoon, Rise of Nations, The Sims, Wii Sports, and Zoo Tycoon. These games are a list of tools that educators can 
choose to employ in their classrooms. Further, the list also provides a list of appropriate games to investigate from a 
research perspective as more evidence to substantiate their use is indispensable. 

We believe that our findings should be used to create a larger validated instrument that can be used on a 
wider audience of educational professionals on the benefits and barriers to integrating COTS video games. This 
paper has documented a dearth of literature on the use of COTS video games in educational settings. Our 
instrumentation is a starting place for researchers to document and validate a tool to collect this pertinent 
information from education professionals. 

As noted by Charsky and Mims (2008), “COTS game are not a panacea, just a strategy with which we, and 
others, have had success; a strategy that may be successful for some educators, especially those educators who are 
already playing video games outside of work.” COTS video games should be perceived as one tool among many 
different instructional strategies - a tool that has the potential to truly engage our students. 
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Introduction and Background 

Technology tools are commonly used in many ways and in various environments in our society. We marvel that 
technology can make our lives more pleasant by helping us easily take care of chores. We can make and take phone 
calls, send and receive text and email messages, and take pictures and send them whether we are in an office, on the 
street, or in a car. We can listen to music anytime and anywhere. The Dick Tracy 2-Way Wrist Radio (and later the 
2-Way Wrist TV) is not just a comic book idea anymore. Technology tools are quickly evolving and changing the 
way we live. 

Our new technology tools enable us to be connected 24 hours a day, blurring the boundaries between our business 
lives and our personal lives (“Available all the time,” 2009) and obscuring the boundaries between public and 
private spaces (Wei & Leung, 1999). We take our mobile (also referred to as cell) phones everywhere. When we are 
at the office, we want to stay connected with family members. When we are with family members, we want to stay 
connected with our business associates. 

As new technologies become part of our culture, it is not always clear what is considered to be appropriate and 
polite behavior. Srivastava (2006) cited a 2003 survey in which half the Americans polled thought that mobile phone 
users were generally discourteous. Another survey from that same year reported that 71% of mobile phone users 
were late for social events because they could easily text or phone to make updated arrangements (Srivastava). 

Emily Post’s rules may be a thing of the past. Faculty at the Wharton School suggest that in the future rules of 
etiquette may be determined by organizations and individuals, with consideration to generational differences about 
the use of the Internet (“Available All the Time,” 2009). 

For example, there are some guidelines that are considered “old rules for the courteous use of a cell phone” (Elgan, 
2010) and include the avoidance of texting or accepting a call during face-to-face conversations unless one has asked 
permission to do so. One’s voice should be lowered when speaking in public spaces and one should avoid personal 
topics if others can hear the conversation. Elgan posits that as cell phone use becomes more ubiquitous and we use 
cell phones in new ways, additional rules may be necessary. For example, Elgan suggests making sure that the 
pictures that are displayed on Facebook are tasteful to all because they could be displayed on a cell phone during a 
business meeting. Elgan also suggests that cell phone users be mindful that tagging people in the Facebook arena 
could cause them embarrassment if those pictures appeared in places that were unintended after they were posted. 
Many schools (including New York City public schools) ban the use of cell phones because they are found to be 
disruptive and promote cheating. However, since the tragedies of September 1 1 and Columbine, school districts are 
reconsidering their usefulness since students bring their cell phones to school so that they can stay in contact with 
their parents (“Cell Phones in the Classroom,” 2010). 

Professors in an urban university have recently noted the unwelcome presence of cell phones in their classes as 
students text each other during class. This silent strumming of the thumbs is considered rude by the professors. Do 
students view this as inappropriate behavior or has this become acceptable among the younger generation? Are 
professors being “old-fashioned” in demanding that the cell phones be stashed and silenced during class? Are the 
rules of etiquette changing? 
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Inappropriate uses of technology abound, regardless of etiquette issues. For example, a professor reports that one of 
her students cited information from the professor’s Web site in a grant proposal (that was funded by the Department 
of Education) without acknowledging the source. Students purchase papers from the Internet, rather than writing the 
papers themselves. The purpose of this research is to share the results of a survey of college students to determine 
their perceptions of appropriate and inappropriate technology use. We explore whether new technology etiquette is 
cultural, generational or gendered. 

The present research builds upon prior work by one of the authors. Rosenfeld and Novemsky (2007) investigated 
professors’ perceptions of inappropriate email use. When asked whether they had received inappropriate email, 
generational differences were found for faculty under age 40 and over age 40, and marginally significant differences 
were found for gender and frequency of receiving inappropriate email. In another study, Rosenfeld and Novemsky 
(2009) surveyed college students and faculty with regard to etiquette and the use of electronic communications. The 
results of that study reported that 68% of the participants thought that it was impolite to multitask in front of another 
person. 

The present survey asks graduate and undergraduate college students questions about their perceptions of 
appropriate/inappropriate technology use. Questions revolve around texting and cell phone use in class and public 
places. We look at gender, age, marital status, education, occupation, student status, ethnicity, and place of birth to 
ascertain if these variables have a negative or positive correlation with student perceptions. 

Participants 

Our survey consists of 129 students in a school of education in a northeastern urban college. Forty-one students are 
on the undergraduate level and 86 are on the graduate level. The students range in age from 20-59 years and are 
grouped in the following age categories: Eighty students (62%) are between 20-29 years old, 27 students (30%) are 
between 30-39 years old, 15 students (12%) are between 40-49 years old, and 7 students are between 50-59 years 
old. The majority of the students were born in the United States (61%), with representation from Middle Central 
America/Caribbean (16%), and South America (8%). Ethnically, 36% are Caucasian, 27% are Black Non-Hispanic, 
12% are Hispanic/Latino, and 12% are Asian. Approximately 57% of the students are single, 12% are single parents, 
16% are married with no children, and 15% are married with children. 

Instrumentation 

Each student was asked to respond to an open-ended "Etiquette Survey" (Rosenfeld & O'Connor-Petmso, 2010). 

The questions consisted of three parts: demographic information, self-attitude rating scale concerning texting and 
cell phone use in class and public places, and cell phone use in the classroom in which they are currently teaching. 

Method 

At the onset of the Spring and Fall 2010 semesters, students were asked to complete a 23 item Etiquette Survey. 
Research questions included items such as whether it is appropriate to text in class during a class discussion, lecture, 
or collaborative work time; whether it is appropriate to text while speaking with a colleague or superior; whether it 
is appropriate to make/take a cell phone call during a meal at a restaurant, during a conference/meeting with an 
instructor or while other people are present, or in a waiting room (such as a dentist’s or doctor’s office); whether it is 
appropriate to discuss personal topics in public places such as the library, a museum and/or restaurant. The teachers 
(n=58) were also asked about their use, their colleagues’ use, and their students’ use of cell phones during the school 
day. 



Analysis 

The data were analyzed using IBM's Statistical Programming of the Social Sciences (SPSS) version 19. Descriptive 
statistics and nonparametric correlations were run to ascertain frequencies and linkages. Common themes emerging 
from the students' open-ended comments on the Etiquette Survey were grouped and reported. 
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Results 



There are no significant linkages between demographics and text and cell phone use and preferences. However, one 
demographic variable, age range, emerges as salient concerning texting in the classroom and cell phone use in public 
places. 

Approximately 44% of the students ranging in age from 20-29 years and 30% of students ranging in age from 30-39 
years believe it is appropriate at times to text in the classroom. The most common reason given is "in 
emergency... urgent message." Additional common comments include "if you are finished with your work," " if your 
colleagues don't mind," "if responses are short," "childcare, family. ...friends," "texting is the only way to stay 
awake," "if the phone is not visible and you can keep your eyes on the professor." 

The age group over 50 years reports that it is inappropriate to text in the classroom. Several students comment that it 
is "disrespectful to the professor," and that they "needed to pay attention." 

Approximately 83% of all students report that it is appropriate at times to use cell phones in restaurants and 76% 
purport that it is acceptable if you are in a waiting room - such as that of a doctor or dentist. Reasons cited for using 
their cell phones are "emergency," "depends on the restaurant and who [sic] you are with," "as long as you are 
quiet," and "if it is a long wait. ..or bored." Approximately 49% of students ranging in age from 20 -29 years report 
that it is appropriate at times, again using the aforementioned reasons, to discuss personal topics on their cell phones 
in public places such as libraries, museums, and restaurants. The other age groups have insignificant or nonexistent 
numbers. 

Out of the 58 (P-K-12) teachers who took the "Etiquette Survey," 97% report that they use their cell phones 
during the school day (primarily lunch time or a break) and relay that approximately 90% of their colleagues use cell 
phones. One teacher complains about her principal "texting during meetings." There are no significant numbers of 
students using cell phones during the school day. 

Discussion & Implications 

The data suggest that there may be generational differences in perceptions of appropriate etiquette for cell phone use 
in classrooms. Whereas 44% of the 20-29 year olds and 30% of the 30-39 year olds feel that there “are times” it is 
acceptable to text in a classroom, none of the participants in the oldest age group think it is ever appropriate. The 
older students feel it is disrespectful and distracting to use cell phones in class. The authors concur with these 
findings, noting that younger students almost always are responsible for the unwelcome texting that occurs in their 
classes. 

Almost half of the 20-29 year olds think it is appropriate to use a cell phone in public places to discuss personal 
topics, whereas the other age groups do not share this perception. Interestingly, there seem to be no other differences 
in perceptions of cell phone use among race, gender, educational level, marital status, or cultural groups. 

Srivastava (2006) suggests that learning to use a new technology can be compared to learning a new language. Just 
as young people pick up a language more quickly than older folks, teenagers appear to be the most avid users of 
technology, including mobile phones. Srivastava points out that because communication technologies have evolved 
so quickly, the ways humans interact with them need to be updated. The guidelines for communicating privately in a 
public space and the appropriateness of being contacted, communicating, and ending a conversation - whether in 
text or in verbal mode in various places - need to be addressed. Established and traditional norms of social behavior 
are challenged by the use of the mobile phone. 

Reaney (2009) reports that rules governing cell phone etiquette are still being formed. Reaney cites an ethnographer 
and an etiquette expert who suggest that cell phone etiquette rules may differ by locality. They propose that in the 
future etiquette rules will have increasing relevance, but what will be deemed appropriate in any given situation will 
likely be challenging to determine. 

The present study seems to reflect these ideas. It is important for all who interact with new technologies to try to 
establish some guidelines for their use since there does not seem to be a consensus regarding appropriate technology 
etiquette. It is apparent that what may be appropriate for one generation may not be perceived in the same way by 
another generation. To avoid conflict in the classroom, guidelines should be given or at the very least, 
appropriateness should be discussed to alert all involved and to maintain a civil and courteous classroom 
atmosphere. 
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Abstract: The use of games for learning is discussed in almost all kinds of educational institution. However, 
empirical evidence for positive effects on learning with these games is still rare. The study described here 
concentrates mainly on the aspect of identification with a character and its influence on learning. Firstly, the 
theoretical basis will be discussed; and secondly, empirical findings from a qualitative study in 2009 are looked at 
and a quantitative study with a learning adventure is addressed. 

Keywords: digital game-based learning, identification, learning, empirical study 

Introduction: 

Serious Games are designed not only to entertain players but also to enhance learning. However, an empirical study 
clarified that the combination of digital games and educational content by no means yields the often postulated 
advantages. The results of the study show that pupils who played a learning adventure did not learn more or better 
from it. Indeed, game players had far worse results than those who dealt with the learning sections of the game alone 
(Seelhammer 2008). Therefore, the question is: which elements of digital learning adventures hinder the learning 
process and which have a positive influence? 

Theoretical Basis: 

One major element is the character which is directly embedded in the game and with whom the player has to work 
cognitively and emotionally. This can lead to identification with the character. The direct implementation of actions 
which are carried out by the player and realized by the character can additionally enhance the relationship between 
them. The enjoyment of identification itself results from the wish to control the relationship with the character. 

Along with para-social interaction and interrelationship, imitation, and affinity, identification is a primarily 
imaginative inner process which leads to an alteration of self-perception while playing (Hefner, Klimmt & Vorderer 
2007), as the player adopts the identity, goals, emotions, and perspectives of the media character. In contrast to para- 
social interaction, identification can be seen as an affiliation of the player with the character. Recipients “become” 
the character, and feel they are part of the game, experiencing feelings like fear or suspense (Cohen 2006). 

As with TV characters, computer players can identify with game characters. A study showed that players ascribed to 
themselves qualities of the played character, such as strength, courage and humor, though the change in self- 
perception was temporary. That change had positive effects on players who had a lower self-esteem before. This 
resulted in a longer abidance in the game, as well as a more extensive examination of the content. In this context, the 
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concept of interactivity - which conveys motivation, informing, understanding, retaining, transfer, and organization 
of the learning process - plays an important role. The appropriate convergence of interactive actions can enhance 
identification because player and character carry out the same operations. When the output does not correspond to 
the input, identification can lessen. As a result, the player is not engaged in the content and may be irritated by the 
lack of corresponding actions. Likewise, the perspective of the character used can influence identification. 
Additionally, it has to be clarified what influence emotions and feelings have on the identification with game 
characters. We would hypothesize that players identify more with characters who have an emotional depth and show 
feelings while performing than with characters that are only “hollow” figures. 

In the same way, the process of motivation can be supported through identification with a character. Players would 
concentrate longer on the (learning) content of the game and, therefore, actively increase their knowledge through 
problem solving and exploration of the game environment. Hence, what kind of influence the process of 
identification has on the motivation to continue staying in the game and the learning process has to be empirically 
proven. 

Research Questions: 

Based on these theoretical considerations, the empirical study investigates the influence of identification with 
characters in learning adventures on the learning process. Current studies concerning identification with characters in 
learning adventures are sparse. One study deals with identification in a first-person shooter game and its influence on 
the behavior of the players (Hefner, Klimmt & Vorderer 2007). However, the possible influence of identification on 
learning was not part of this or any other study. 

Concerning these assumptions, two main research questions should be revised. 

1. What kind of influence does the identification with a character of a learning game have on the learning 
process? 

2. What kind of influence does the identification with a character of a learning game have on the motivation to 
continue playing the game? 

Empirical Study: 

The first empirical findings were gathered from a qualitative study in September, 2009, where girls, aged 10-14, 
played a mathematical learning adventure. The girls were observed and afterwards guideline interviewed. The 
recorded and transcribed statements were researched for similarities on identification, learning, interest in 
mathematics, and the game itself. Results concerning the character showed that the girls talked with their characters 
and referred to them, mostly female, as “I”. That is an implication for a development of a relationship between player 
and character. In the first half of 2010, there will be a quantitative study with the learning game “Tong Pak Fu and 
Chou Heung”, which is a probability fantasy. Thereafter, this learning content will be put into another gaming 
situation with changing main characters from which the player can choose, in order to have a look at identification 
and its influence on learning. The first results from this are expected at the end of the summer, 2010. 



268 




Goal: 

The aim of this research is to give designers and authors guidelines on how to create and design the main characters 
of learning games with which the players can identify. If gamers can identify themselves with characters, the 
following goals can be reached: efficient learning, escalation of motivation, continued learning with the game, more 
time spent in the learning environment, and active use of new knowledge which can be transferred to the real world. 
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Abstract 

This research was designed to explore the impact of either backward fading or collaboration on fraction 
learning of children with special needs. In order to perform the given tasks, a computer software DigiTile, which has 
the ability to serve multiuser interface with one computer, were provided to children. Analysis of quantitative data 
collected through pre and post tests and observation sheets indicated that backward fading and collaboration groups 
in the study demonstrated significantly better performances when compared with the control group. In addition, 
Collaboration group also outperformed backward fading group. These results also supported by the interviews 
conducted with the participants. 



Introduction 

Instructional settings have been ever exposed to new technological advances like computers. After 1995, 
because of increased availability of microcomputers and the Internet, computers as instructional media became 
prevalent (Reiser, 2002). In the literature, there are studies exemplifying the significance and no-significance of 
computers as instructional media. While studies mainly focus on integration of computers into mainstream school 
settings, some research deals with integration of them into settings for children with special needs (CSN). Studies 
indicate that these children seem to profit from the use of computers in the areas of recognition, support and the 
enhancement of motivation, self-confidence, and self-esteem (Keates, Clarkson, &Robinson, 2002). Also, they enjoy 
playing and experience a sense of solidarity at the computer (Brodin & Lindstrand, 2000). 

It could be essential to increase participation among children who find speaking difficult or who are unable 
to verbally contribute to group settings. In order to increase this participation, the environment should be designed to 
principles of collaboration defined as “coordinated, synchronous activity that is a result of a continued attempt to 
construct and maintain a shared conception of a problem” (Roschelle & Teasley, 1995, p. 70). Because of interactive 
features, computers can enable educators to set collaborative learning environments. The goal of computer supported 
collaborative learning (CSCL) is mainly provide environments to enhance contribution of students for mutual 
knowledge gaining and sharing (Prinsen, 2009). 

The design of the instructional messages is very crucial when limited capacity of working memory is 
considered. Cognitive load theory (CLT) emerged to suggest certain guidelines showing how to diminish cognitive 
overload. It basically offers series of evidence-based guidelines to make use of maximum capacity of mind for 
learning (Clark, Nguyen, & Sweller, 2006). Guideline 19 written by those authors is related to gradual learning 
process by introducing worked example, carrying on with completion example, and then practicing with full 
problem. Authors called this process “transition from worked examples to problem assignments with backwards 
fading” (Clark et al., 2006, p. 197). 

In the light of this, the focus of the study is to investigate the effect of “backward fading” and CSCL 
environment through the use of a fraction learning software DigiTile. To achieve this, the following research 
questions will be investigated; 

• Do post-test scores differ for CSN who attended backward fading condition, who attended control 
condition, and who attended collaboration condition assuming no prior differences in the pre-test 
scores among groups? 

• Is there any relationship between post-test scores and the number of correctly completed tasks? 
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• Is there any relationship between total time spent to complete tasks and involvement of one of the 
interventions? 

• What are perceptions of CSN about collaboration or backward-fading technique or working with 
computers? 



Methodology 

This research has a quasi-experimental design with both qualitative and quantitative nature. The data 
collected by the use of pre and post-tests was analyzed by quantitative techniques, and collection and analyze of 
voice recording constitutes qualitative part of the study. 

54 children participated to the study. There are 24 female and 30 male students. Among them, 16 students 
were included in collaborative intervention; 19 students were in backward fading condition; and the rest were control 
group without any collaboration and any fading. These children are attending a school for CSN. They mainly have 
learning difficulties at different levels. In addition, they are joining regular classroom activities of mainstream 
schools. To determine which students are able to complete tasks given during the study, their teachers helped about 
their learning difficulties. 

Four instruments were used for data collection: (i) pre & post tests; (ii) tasks; (iii) observation sheet; (iv) 
interview guideline. Pre-test and post-test are the instruments for describing existing fraction knowledge before and 
after the use of DigiTile. Questions assess specifically equivalence and comparison of fraction numbers. 

The second instrument is composed of tasks designed to be performed by DigiTile. There are 6 tasks 
altogether in two sets; 3 for fraction comparison in one set and 3 for equivalent fractions in the other set. Each set in 
backward fading condition has three levels: worked example, completion task and full practice. Other two groups 
engaged only full practice either alone or collaboratively. 

The third instalment is the observation sheet including different check points showing trials, attention, 
confusion, time, interaction, correctness, and collaboration. Number of the trials a student makes before finding a 
correct answer or giving up trying is counted by use of this observation sheet. Observer is also enabled by this sheet 
to make comments and give grades on student attention, interaction and confusion during the activity, and trace the 
evidences of occurrence of collaboration among peers. The completion time and correctness of students is also 
evaluated via observation sheet. The last instrument is the interview guideline designed to find out students’ 
perceptions about collaboration feeling about task difficulty. 

DigiTile is developed for children to help their understanding about fractions. It is a program to help 
through motivation and playful learning. It helps children to form representations about fractions that mediate their 
thinking. The importance of computer supported collaborative learning is to include instant feedback, self-control 
stimuli, bright colors and sounds and interactive aspects. DigiTile is an application which has these characteristics 
apart from sound and poses tasks for children and also includes instant feedback, self-control stimuli, bright colors 
and interactive aspects and poses tasks for children. It is computer software combining interactive learning 
environment with collaboration which aims to enhance understanding of fractions, proportions and symmetry. It 
requires children to move mosaic tiles onto a grid to complete given challenges. The feedback for fractions appears 
in the areas at the sides of the table while children are moving tiles. Children were given tasks to complete using 
DigiTile while working alone or with each other. 

Procedures 

One week before intervention, students were given pre-test and training about use of DigiTile. Backward 
fading group was first shown a worked example performed in DigiTile. Then they were asked to complete one step, 
and finally to make full problem solving. Meanwhile, control group was only given a full practice task. Another 
group (collaboration group) was asked to complete a full practice activity but at this time students tried to complete 
the task with their peers. Table 1 summarizes procedures of the data collection. While students were working on the 
tasks, two researchers rated their performance-related data and they also took some field notes. However, for this 
study only some parts of these data were used because of time limitations. Each session was voice recorded. 
Interviews took place after the termination of each session to learn their perceptions about the collaboration and/or 
DigiTile. 

Table 1 

Summary ; of data collection procedure 
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Backward fading 


Single Control 


Collaboration 


Single performance 


Single performance 


Peer performance 


Worked example 


Full practice 


Full practice 


Completion example 






Full practice 







Results 

Collected quantitative data was analyzed with the utilization of SPSS 15.0. Analysis of covariance 
(ANCOVA) was run to understand the effects of either CSCL or backward fading on the fraction learning of CSN by 
taking pre-test scores as constant. Pre-test scores were taken as covariate because the starting knowledge levels and 
academic backgrounds were diverse, thus their gained knowledge were analyzed in this way. To make sure the 
appropriateness of the data analysis model, all assumptions were checked and no violations were found in terms of 
normality, homogeneity of variance, outliers, linear relationship, and homogeneity of regression coefficient. No 
interaction was detected between covariate and dependent variable [F(2, 48)=. 38, p>.05]. Results of the analysis 
showed that being in one of the intervention groups [F(2,50) = 28.53 , p=.000, r\ 2 = .53] or scores of pre-test [F(l,50) 
= 147.83 , p=.000, r) 2 = .75] have significant effect on scores of post-test. Effect sizes offer large values (Field, 2005) 
meaning that 53% of variance was explained by the mean difference of intervention groups if pre-test scores were 
considered as constant. There is a summary of results in Table 2. 

We applied Sidak correction for confidence interval adjustment. Table 3 summarizes the results of this 
correction. It is seen that there is significant mean differences of post-test scores between groups. To explore the 
effect of interventions, alpha was adjusted at the level of .025 and Bonferronni procedure was utilized to understand 
contrast analysis results. Like Sidak test, Bonferronni produced significant differences between intervention types in 
terms of post-test results (Table 3), too. 

Results of the ANCOVA obviously indicate that there are meaningful mean differences of post-test scores 
of participants involved one of three conditions after controlling their pre-test scores. As pairwise comparisons 
revealed, all of each two conditions have also significant mean difference. Collaboration condition group had higher 
post-test scores than either single control group or backward fading group. On the other hand, backward fading group 
did better on post-test than single control group did. 



Table 2 

Summary Table of ANCOVA Results with Dependent Variable of Post-test Scores With Pre-test Scores 



Source 


SS 


df 


MS 


F 


tf 


Covariate 


105.56 


1 


105.56 


147.83* 


.75 


Between 


40.74 


2 


20.37 


28.53* 


.53 


Error (Within) 


35.70 


50 


.71 






Total 


609.75 


54 









*p< .05, 1) =effect size 



Table-3 

Pairwise Comparisons with Sidak Adjustment for Multiple Comparisons 



Intervention Intervention MD SE Sig 
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Backward Fading 


Backward Fading 


1.33* 


.29 


.00 




Control 


-.80* 


.29 


.03 


Control 


Backward Fading 


-1.33* 


.29 


.00 




Collaboration 


-2.13* 


.30 


.00 


Collaboration 


Backward Fading 


.80* 


.29 


.03 




Control 


2.13* 


.30 


.00 



*p < .05 

Results of the ANCOVA obviously indicate that there are meaningful mean differences of post-test scores 
of participants involved one of three conditions after controlling their pre-test scores. As pairwise comparisons 
revealed, all of each two conditions have also significant mean difference. Collaboration condition group had higher 
post-test scores than either single control group or backward fading group. On the other hand, backward fading group 
did better on post-test than single control group did. 

As further analysis, we ran a correlation between post-test scores of CSN and correctly completed task 
amount. The relationship was found significant (r =.42, p<.01), i.e. as the number of successfully completed tasks 
increased, CSN’s post-test scores on geometry also increased (Table 4). 

Table 4 

Correlation Table Between Post-Test Scores of CSN and Correctly C ompleted Task Amount 





Posttest 


Correctness 


Correctness Pearson 


42 (**) 


1 


Correlation 


Sig. (2-tailed) 


.001 




N 


54 


54 



** Correlation is significant at the 0.01 level (2-tailed). 



To reach a deeper understanding for the results of quantitative analysis, qualitative data collected through 
the interviews were also analyzed. In general, except for five students (two from control group, two from 
collaboration group, and one from backward fading group) found the activities easy. On the other hand, two students 
from control group expressed the difficulty level as moderately difficult. Only one student thought the activities were 
moderately easy for him. 

During the study, students were exposed two different tasks about comparison and equivalence of fractions. 
When students were asked which type of the activity is hardest and easiest, equivalence of fractions tasks was seen as 
the hardest, and comparison of fractions tasks was seen as easiest. 

Motivation of CSN is an important element in terms of success of the study. Working with computers and 
its multimedia abilities provides researchers with plenty of opportunities to motivate children in such studies. It was 
also approved by the children interviewed. They expressed their positive opinions about the interventions because 
studying with computers prevented them to get bored while studying. 

There was a distinction between students working with peers and working alone in terms of definition of 
their feelings about DigiTile. Students working with pairs used words like “fun”, “enjoyment” and “entertaining”. 
They also found working with pair enjoyable and just one student expressed his preference to work alone. On the 
other hand, students working alone describe their feelings around the word “computer”. For instance, one of them 
said “I will use this program if I had a computer in my home”, and one other said “it was nice to use computer and 
playing with colors in the activities”. They were also judgmental and task oriented when answering the question 
about overall thought about the study. Especially children in backward fading group generally expressed ease-of- 
doing. The followings are the comments of two students: 

I completed tasks although they were difficult. 

1 finished all the tasks because they were very easy. 

They thought it was positive for them if tasks were firstly performed by the teacher. Meanwhile, children 
from control group did not even motioned about the tasks and processes; they only said that using computers were 
enjoyable. 
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Discussion and Conclusion 



Findings obtained through the study demonstrated that backward fading and collaboration are significant 
contributors to fraction learning of CSN. Studies comparing backward fading and problem solving generally exist in 
the literature. Our study can be regarded as similar since given frill practice tasks have the nature of problem solving. 
Although there are some researches indicating superior effects of backward fading when compared other variables 
like problem solving or forward fading (Renkl & Atkinson, 2003; Renkl, Atkinson, Maier, & Staley, 2002), in some 
situations like far transfer, any difference between backward fading and other methods was not observed 
(Moreno, Reisslein, & Ozogul, 2009). 

According to the findings, collaboration group outperformed backward fading group. Three possible reasons 
can be mentioned about this situation. Firstly, students in the collaboration groups might be provided with 
scaffolding by their peers. Secondly, testing and re-testing children in one week period could manipulate the test 
results. That is, if post-test was conducted after more than one week period, more reliable comparisons among the 
post-test results of the groups could be gained. The reason is that backward fading might not produce close results 
when compared problem solving in terms of near transfer but in terms of far transfer (Moreno, Reisslein, & Ozogul, 
2009). 

Thirdly, another possible explanation could be related with the difficulty levels of the given tasks. For both 
group of the comparison and equivalence tasks, we firstly assigned easier task, and then harder one. The increasing 
task difficulty may have side effects on the data collected through the pre and post tests. According to the study 
conducted by Smith (1999), students made more mistakes when they given whole task training, but when task 
difficulties were kept constant, increased mistakes were observed in backward fading training. Although this result is 
not parallel with our finding, data obtained from interviews directs us a similar conclusion. Nearly all of the children 
from backward fading group found activities easy, and they do not indicate the sense of changing difficulty levels 
through activities. So, as Smith resulted, perception of constant level of task difficulty might lead backward fading 
group get relatively lower scores from post-test. The result yields the need for more effort to find out which group 
made more mistakes in which task. 

Our findings about the existence of a relationship between involvement to one of the interventions and total 
time spent is in line with the results of Paas & Van Merrienboer’s (1994) study about worked examples first 
vs. conventional problems first conditions. Similar to our results, the former condition was found requiring less time. 
Backward fading also seem to have worked well for these students with special needs. Weber (1978) found the 
positive effect of backward chaining studying with mentally retarded adults. This study is also an example of the 
advantage of this technique with respect to time. 

Another prominent result indicated by the research is that collaboration is effective in fraction learning with 
CSN. In today’s classes, predominantly single user computers are being used. Single use of computers limits children 
because the control is only in one user. This leads to frustration because not all children can interact with computers 
equally (Stewart, J., E. Rayborn, B. Bederson, A. Drain, 1998). Sometimes it causes discussions among children 
about turn taking issues. Also, the others may find it difficult to follow what the child in control is doing. This is 
especially an important issue for CSN because of the difficulty to maintain attention in an activity. They are often 
distracted easily and unable to sit long enough to complete a task. If they cannot find the opportunity to interact with 
the display, they may not maintain their attention on the task hence it limits their learning. Using two mice in this 
study eliminated the control of one child. Both children had the opportunity to interact with the display and make 
changes on it simultaneously. This led to increase in collaboration and incorporation of the child to the task those 
enhanced the fraction learning. 

Furthermore, children with learning difficulties, autistic spectrum disorders, speech and language problems 
or who are shy or stutter may have passive role in single user display because they cannot contribute verbally. 
Flowever, in a shareable interface they may find the opportunity to interact with the display without speaking which 
makes them active learners (Rogers, Lim, Flazlewood and Marshall, 2008). Parallel with this, interviews with CSN 
from collaboration group indicated similarly that using computers (in a sharable environment with two mouse) 
during tasks is fun although they generally avoid interacting with their peers by speaking. 

Also, these results supported the finding that being aware of what each other is doing is an important 
determinant in successful collaboration (Dourish and Bellotti, 1992, cited in Rogers and Lindley, 2004) which result 
in better learning. It can be concluded that collaboration is beneficial for CSN. This is because both their academic 
and social needs are addressed (Flasselbring, 2000). Also, research has shown that students of all ability levels learn 
more when they are involved in such knowledge construction activities (Collins, Flawkins and Carver, 1991). 
Children who have learning and attention difficulties are supported in a collaborative learning environment where 
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they can have an active role and feel a part of the task. These activities may support equal opportunities for children, 
social skills and learning. 

Computer-based activities those have visual activities and feedback mechanism are used to increase 
children’s attention to a task (Cardona et.al., 2000). In this study, DigiTile provide the children a visual and colorful 
representation of fractions which led to attract their attention to the task. Positive effect of visually appealing 
computerized environment on attention was supported by the interview results with CSN. CSN needs seem to enjoy 
playing and experience a sense of solidarity at the computer (Brodin and Lindstrand, 2000). Our findings from the 
interviews support these studies, because students frequently brought on the computer related enjoyment. Since such 
perceptions could have increased their motivations, further data can be included for further dimensions of the study. 

In conclusion, this study underlines positive effects of collaboration and backward fading on fraction 
learning of CSN in a computerized environment. It also highlights the way for researches about possible effects of 
these methods for learning of subjects other than fraction learning. Working with children with different mental 
disorders is also necessary to reach more generalizable findings. There are some limitations which probably affected 
the reliability of the results. Short period between pre and post-tests is the most important limitation of the study. In 
addition, lack of a retention test, relatively inadequate number of participants can be counted as barriers preventing 
the researchers to conclude more generalizable results. 
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Abstract 

This study aims to evaluate Web-Based Internet Search Scaffolding Tool (WISST) that has been developed 
to help students search purposefully in the Internet through certain scaffolding strategies implemented in the tool. 45 
6 th , 7 th and 8 th grade students and 4 experts participated in this study. The data were collected through focus group 
method, the target groups’ perceived usability problems were detected and their solutions were gathered 



Introduction 

Children constitute an important part of current Internet users. Literature shows that they generally use the 
Web as an information source especially for school assignments (Large, Beheshti, Nesset, & Bowler, 2004; Large & 
Beheshti, 2005). While engaging in search tasks within the Web environment, children usually confront with adult 
specific interfaces. Therefore, interaction can turn into a maze. This commonly happens in the utilization of such 
search engines as Google because they serve for general purpose searches. The technology behind such tools does 
not matter if the results do not appeal or satisfy the user (Chau, Qin, Zhou, Tseng, & Chen, 2008). In addition to the 
tool itself, there are other factors influencing the search performance of users. The initiation and the flow of the 
search process depend on the user’s understanding of the information they are requested (Lucas & Topi, 2004). For 
example, novice users generally do not plan before starting, and they visit the links in the list they find without 
evaluating them critically (Quintana, Zhang, & Krajcik, 2005). Because of that it is essential to guide them in the 
search process. 

Searching the Internet, which requires decisions to be made to access relevant information, does not always 
generate intended results (Fang & Salvendy, 2000). Use of metacognitive skills is essential to examine high density 
information in the Internet. Especially primary school children face certain difficulties in this process. As a result of 
learner-centered paradigm, performance assignments requiring Internet search are given to the students frequently. 
However, providing guidance and scaffolding for children and helping them use their metacognitive skills throughout 
the search process is essential (Sendurur & Yildirim, 2008). In this study, to provide support for the elementary 
school students in the search process a “Web-based Internet Search Scaffolding Tool” (WISST) developed. WISST 
was designed in the light of Quintana et al.’s (2005) framework. This tool aims to scaffold students throughout Web 
searching process by emphasizing certain metacognitive skills improvement with the help of “asking question”, 
“searching”, “evaluating and reading”, and “synthesizing” steps. In this study, only the first usability evaluation 
process and related findings will be presented. 

In this study, first, it is aimed to find out common Internet search habits of target users. Then, development 
and user-centered iterative evaluation processes of WISST, target users’ recommendations for improving WISST, 
and their perceived usability problems of the tool were presented. 

Method 

Iterative design approach was used in designing WISST. Qualitative methods were used to collect the 
relevant data. Usability evaluation of WISST was conducted with the primary users. WISST was designed for 6 th to 
8 th grade students. Therefore, 45 6 th , 7 th , and 8 th grade volunteers (15 students from each grade level) from one of the 
elementary school in Ankara, Turkey participated in this study. Even though the real users provide valuable feedback 
in the design process, they are not the designers (Oosterholt, Kusano, & Vries, 1996). In order to understand the 
target users’ opinions, focus group interviews were conducted. First, students were asked to describe how they search 
the Internet. Then, the interface of WISST was projected and students were allowed to discover the functions of 
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elements in the interface. After being informed of the real functions, they discussed what to be changed and drew or 
wrote their suggestion on cardboards. The interview sessions were tape-recorded. 



WISST 



According to Quintana et al. (2005), through an online inquiry, children experience certain cognitive phases. 
The inquiry starts with asking questions, then the searching occurs, and then evaluating and reading comes. 
Synthesizing is the final cognitive step. The authors suggested that those steps can be scaffolded by certain 
approaches. Web-based scaffolding approach utilized in WISST focused on these approaches. According to 
suggested framework some of the metacognitive strategies applied in designing WISST can be summarized as 
follows (Quintana et al., 2005, p. 237): 

• Asking question: Provide driving questions; help to integrate results of multiple searches in one space. 

• Searching: Encourage users to find rich resources; make search steps visible; help users to decide on 
keywords before searching; show the search history. 

• Evaluating and reading: provide a prompted notepad; show users their goals; provide users with a list of 
evaluation criteria. 

• Synthesizing: Encourage users to compare and contrast information across different resources; describe 
the criteria they should use; prompt users to reflect on different aspects of information. 



Procedure and Data Collection 

Iterative design begins with design, testing and measuring, and continues with redesign, retest and re- 
measuring. These steps continue in a loop and stop when the satisfactory results are gathered (Shackel, 1991). Since 
the final product will be used in real educational context, the formative evaluation was conducted in a public school. 
The iterative design will continue until the usability evaluations results indicate satisfactory findings. In this study, 
only the first usability evaluation process will be described. 

The focus group sessions took place in a public school in Ankara, Turkey. In this school, there are two 
computer laboratories. 45 volunteers participated in the study and three focus group sessions one for each grade level 
was conducted. Each focus group lasted two 40-minutes consecutive sessions. The interface of WISST was 
projected on a screen. First, the students were asked to describe the way they follow to complete their performance 
homework. Then, without explaining the overall functions and aims of the software, they were allowed to discuss 
and find usability issues, problems they face, and their preferences of the program with peers by facilitation of the 
researcher. Throughout the sessions, the researcher explained the function of software, helping them about the use or 
aim of the program, and answering children’s questions. The software was only projected on a screen. They were 
asked to express their first impressions and irritated items or functions. In the second session, blank cardboards were 
given to the students, and they were encouraged to draw or write their preferences of the software, and suggestions to 
solve the perceived usability problems. The sessions were tape recorded. 

Data Analysis 

The focus group session records were transcribed, analyzed, and coded to diagnose users’ Internet search 
habits, and perceived usability problems about WISST prototype. The data gathered through group sessions were 
supported through analysis of drawings or essays of the students. In addition, those data were used to diagnose the 
solutions suggested by the target users by calculating the frequencies and determining the common concepts reported 
by the students. 



Findings 



Internet Search Habits 

Preferences: All participants conduct Internet search at least two times in a week to complete their 
performance homework. However, 3 students in sixth grade do not have a computer at home. The remaining students 
have computers at home, and use it actively. The most preferred search engine by the students is Google because of 
its ease of use and option provided for Turkish search. 
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Experienced Difficulties: All groups complained about broken links in the Web. Both 6 th and 7 th graders 
mentioned about the difficulty in accessing specific information due to misleading links or irrelevant list of results. In 
addition, 6 th graders stated that they were confused easily and lost frequently within the result list. 

Keywords: All participants found that deciding the keywords was easy. They generally use more than one 
keyword at once and just put a space between them. 

Search Patterns: Without any exception, all target users stated that they start searching by extracting 
keywords from the performance homework question. Then, they enter those keywords in Google. When results are 
listed: (i) All 6 th graders and 4 8 th grade students read explanations under links one by one in the given order; (ii) All 
7 th graders and the majority (9 students) of the 8 lh graders reported that they scan the titles; (iii) 2 8 th grade students 
just click the first link. When they investigate the sites their strategies do not differ, i.e. they all benefit from titles on 
the pages. If the titles are appropriate, they continue to read. In addition, 7 th graders use the titles as a reliability 
criterion. They indicate that if the site includes all the needed titles (the same with or similar to those in the 
homework), it can be inferred that the site is reliable. While 6 th graders have no criteria for reliability of sites, 8 lh 
graders prefer to compare one site with another, and if the sites provide the same information, they mention that it is 
a sign for reliability of that site. Finally, all 6 th graders prefer direct copying and pasting strategy to complete the 
work, but half of 7 th and 8 th graders stated that they generally feel that it is needed to add certain explanations to the 
copied and pasted information. 

Perceived Usability Problems 

Almost all users’ guesses about the aim of the software were on point or close to the point. However, the 
menu titled “Arama” (search) revealed some problems. Five 6 th graders anticipated that it was just a title. The rest of 
those group claimed that it allowed managing account options. Despite guessing the functions of the menu correctly, 
other groups pointed the visibility problem of the menu. Figure 1 demonstrates close and open positions of “Arama” 
menu. 



| SB Odev Ara$tirma jzleyicisi SB.*, 




Another perceived problem by the target users is related with pop-up windows requiring user input. Except 
for majority of 7 th and 8th graders, the rest of the students found the questions posed by the tool too many and 
unnecessary. They stated that they did not need those questions. Indeed, 6 th graders directly stated that copying and 
pasting do not require such questions, thus there is no need to think about the details. However, they also mentioned 
that it could be helpful if the sites were investigated well. Majority of all groups also stated that some questions in the 
termination part could help to judge the quality of the information in the site. An example of pop-up window and the 
questions can be seen in Figure 2. 
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Figure 2 

“Alinti Yap” (quotation) option of menu was also perceived as a problem by target users. Majority of 6 th 
graders, two students from 7 th grade and one student from 8 th grade found “quotation” in this software difficult since 
they already do not either add any additional idea or reexamine previous quotations. Other users, on the other hand, 
found using “quotation” easier because they do not need to minimize the search window, and then to open or to 
maximize another document. In this way, time could be saved as indicated by some 8 th graders. 

The Participants’ Recommendations 



The participants’ recommendations were as follows: 

• 34 users suggested color modifications: 14 users favored blue; 12 users green; 8 users red. 

• 14 users suggested icon-based or button-based horizontal version of menu in order to make accessibility 
of “start” and “finish” types of functions easier and visible. 

• 12 users wanted the inclusion of some visuals like cartoon characters. 

• 8 users proposed that the left panel should be moved top or bottom of the screen because of the demand 
for large search space. 

• 7 users pointed the need for a space for note taking in addition to quotation part. 

• 7 users demanded certain amount of customization of their accounts. 

• 5 users wanted to change the fonts, styles, and colors of texts. 

• 2 users suggested a help option. 



Discussion 

The findings clearly imply that design of software is not or should not be a one-shot process. Especially, 
involvement of target users in the early stages of design contributes to decide on what to do next with the interface. 
By considering that importance, the process started with analysis of users’ Internet search habits. Although there are 
some slight differences between different grade levels, a common search pattern of the participants can be inferred 
from the findings of this study. They generally do not prefer to conduct detailed search. Most of them stop when they 
find all in one no matter whether it is the first Web site visited. However, some students who are either 7 th or 8 th 
graders reported that in order to be sure whether that information is adequate or not, they visit a second site and if the 
titles or information included are the same with the previous one, then they decide on terminating the search process. 
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All students face some difficulties during searching (Bar-llan & Belous, 2007). Findings suggested that broken links 
are the most common problems the students face. Unlike other levels, 6 th graders experience more confusion during 
Internet search. This might be related with the amount of previous experience. On the other hand, they all search for 
ready-to-use and specific information, but this is considerably hard for especially 6 th graders. This finding is 
consistent with previous research done regardless of age (Chau et ah, 2008; Large & Beheshti, 2000). Titles and 
explanations serve as visual cues about the information or website for the majority. Based on those, they decide on 
which ways to go in their search. For completion of performance homework, they generally prefer copy-paste 
strategy. Unlike 6 th graders, about half of the other groups sometimes add their own words. It can be concluded based 
on the findings that the amount of previous experience affects the patterns of Internet search, and that might be the 
reason for why 6th graders’ answers differ from the others. 

Focus group analysis revealed that the appearance and functionality of menu have certain problems. 

Students were irritated with the placement and background of it. In other words, they perceived it as just a title 
because of being located on a blue bar. Some students found the questions on the pop-up windows unnecessary and 
time consuming. This is because they just start searching by entering keywords and continue with copy-paste. 
However, others who prefer to organize or add something of their own found those questions as a checklist to 
control. “Quotation” function of menu was also perceived as a problem by especially 6 th graders, but others found it 
easier due to elimination of a few steps. 



Conclusion 

This study explained the first usability evaluation process of WISST prototype. Focus group methodology 
allowed understanding the Internet search habits and strategies of target users. Moreover, users’ suggestions provided 
valuable feedback especially about cosmetic features of the prototype as well as perceptions of problematic features. 
Based on the findings of this study, the tool will be improved, and then another usability evaluation will be 
conducted with the improved tool. While conducting this usability evaluation, the functions of the tool were limited, 
and this resulted with lack of hands on user testing. Considering this limitation, for the next iteration, first the target 
group can use the improved tool to search for a performance-homework, and then focus group sessions can be 
conducted. This process can provide rich data about the tool and its functions that will lead to develop better Web- 
Based Internet Search Scaffolding Tool. 
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Introduction 

Csikszentmihalyi (2006) noted “creativity is no longer a luxury for the few, but ... a necessity for all” (p. 
xviii). Learning organizations appear to agree, as a variety of 21 st Century Learning Skills (North Central Regional 
Education Laboratory and the Partnership for 21 st Century Skills) have included concepts such as “thinking a 
problem-solving skills,” “inventive thinking,” “high productivity,” “using digital technology and communication 
tools,” and “learning academic content through real-world examples” in their standards. In Tough Choices or Tough 
Times from the National Center on Education and the Economy, McWilliam (2008) explained “the 21 st century is a 
world in which comfort with ideas and abstractions is the passport to a good job, in which creativity and innovation 
are the keys to the good life, in which high levels of education - a very different kind of education than most of us 
have had - are going to be the only security there is” (p. 67). 

Innovation Partnerships for Teachers (IPT) is one of a three-component NSF-sponsored project which 
supports an online course designed for teachers to engage in pedagogies and technologies associated with creativity, 
innovation, and invention. The other two components of the grant include the Manufacturers Innovation Leadership 
Program (MILP) to connect leader-managers from small- and medium-sized manufacturing (SMM) firms with 
university scholars and K12 teachers and the Innovation Scholars Program to create multi-disciplinary teams of 
senior college students and faculty who will produce and process innovation projects with small manufacturers. 

Conceptual Framework 

The conceptual framework upon which the IPT project is based was developed in 1995 through the pioneer 
efforts of the Jerome and Dorothy Lemelson Center for the Study of Invention and Innovation (Smithsonian 
Institution, 2006). The Center, part of the National Museum of American History of the Smithsonian Institution, 
focused its attention on developing programs aimed at youth interested in the study and exploration of innovation 
and invention and its impact on American society. Within that context, the IPT project focuses on the interaction 
among teachers, students and experienced business practioners, each of whom, learning together, understand that 
innovation begins with the learning process of creative problem solving. Specifically, IPT develops an Internet- 
delivered pedagogy course for K12 teacher to enhance their understanding of and abilities to design, develop, and 
deliver rich lessons that engage students in creativity and innovation technologies. The course features authentic 
issues in teachers’ and students’ day-to-day lives. This is significant, because 21 st Century Skills for learning call for 
students understanding how to live and work in a global economy, and innovation technologies and creativity open 
exciting career opportunities in such an economy. 

IPT places what students learn in the classroom within the broader context of the economics of real life. 
Course activities are designed using the following tools: 

• National learning standards for appropriate subjects and grade levels, 

• Partnerships with small- and medium-sized manufacturers (SMMs), 

• Team-based innovation problem solving that addresses creativity in new product development, 

• Business, engineering, and communications issues associated with innovation, 

• Design and rapid prototyping, and 

• Partnerships with university faculty and university students who will be available as resources to mentor 

K12 students as they learn the innovation process. 

It is this process - innovation, that is essential for living and succeeding in a global society. Through 
innovation, students also learn that manufacturing within the global economy is an exciting and attractive career 
choice in their professional futures. This is the same realization that causes business communities to be attracted to 
schools, offering expertise and financial support. 
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Program Objectives 



The objectives of the IPT program include: 

1. Develop a three-credit hour Master’s level web-based course in pedagogy that emphasizes creativity and 
innovation in solving ill-structured manufacturing problems for K-12 teachers. 

2. Each participating teacher will produce two authentic, problem-based lesson plans using innovation 
technologies and manufacturing concepts to use in their classrooms and share with other teachers via the 
project website. 

3. Encourage the creation of school-manufacturing business partnerships to further creativity and innovation 
technologies in teaching and learning. 

4. Impact teachers’ attitudes and awareness of teaching creativity and innovation technologies and 
manufacturing as a significant economic force and career choice. 

5. Use Design-Based Research Methodology to assess the project’s design, development, and effectiveness. 

Defining Creativity and Innovation 

Within the respective disciplines in which the concept of creative or creativity is employed, researchers 
have attempted to define the concept relative to the discipline itself. Markman (2009), co-editor of the book Tools 
for Innovation, indicates that creativity is using language to speak sentences that have never been spoken before, or 
expressing thoughts that have never been expressed. Creativity within the science and engineering disciplines 
includes discoveries of new knowledge in science and medicine, invention of new technology, composing beautiful 
music, or analyzing a situation in law, philosophy, or history in a new way (Standler 1998). Finally, creativity in 
education has been referred to as the "ability to produce work that is both novel and appropriate" (Sternberg & 
Lubart, 1999). Craft (2005) views creativity as the ability to see possibilities that others haven't noticed, while 
Esquivel (1995) explains it as the critical process involved in the generation of new ideas. In an operationally 
problem-oriented definition, Torrance (1962) defined creativity as: A process of becoming sensitive to a problem, 
deficiencies, gaps in knowledge, missing elements, disharmonies, and so on; identifying the difficulty; searching for 
solutions, making guesses, or formulating hypothesis about these deficiencies; testing and retesting these hypotheses 
and possibly modifying and retesting them; and finally communicating the results. Ultimately, regardless of how the 
particular discipline defines creativity, it is essentially recognized as how best practice and imaginative ideas can be 
applied in diverse situations. 

The critical or defining attributes of creativity were best described in a classic study by Taylor and 
Williams (1966), which proposed the existence of five typologies for creativity. These were expressive, productive, 
inventive, innovative, and emergenative. They explain that expressive creativity is spontaneous and often observed 
in children while scientists or artists generally illustrate productive creativity. Problem solving or creation, such as 
improving technology is an example of inventive creativity. Innovative creativity requires the capacity to improve, 
invent, or reinvent an existing object through conceptualization skills. Finally, emergenative creativity discovers or 
opens a surge of ideas to explore and develop new inventions. 

What makes a person employable in our digital, globalized society? McWilliam (2008) found that the key 
factors for employability of 21 st century workers include: 

• Personal attributes (such as motivation and adaptability) 

• Communication (such as emphasizing and speaking non-English languages) 

• Teamwork 

• Initiative and enterprise (including ‘being creative’) 

• Problem solving (such as developing creative, innovative solutions) 

• Planning and organizing 

• Self-management (including intellectual property 

• Life-long learning 

• Familiarity with technology 

The call for a more creative workforce rings loudly in politics and society (Seltzer and Bentley 1999; Leadbeater, 
1999; Landry, 2000; Florida, 2002). The role digitally enhanced environments play in fostering creativity have 
generated a great deal of research (Cunningham, 2002; Hartley, 2004; Howkins, 2001; Caves, 2000). Pink (2006) 
details how our future workers will be performing work that is focused on creative solutions to problems and 
synthesis of ideas and concepts. 
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According to research, creative and innovative people possess the characteristics and dispositions necessary 
to be this type of employee: intrinsic motivation (Amabile 1983, Deci & Ryan 1985, Hennessey 1999, Hennessey & 
Amabile 1998, Urdan 1997, and Henderson 2004); risk taking (Perkins 1981, McWilliam 2008); pattern recognition 
(Perkins 1981); exploration (Henderson 2004, McWilliam 2008, Perkins 1981); problem solving (Perkins 1981, 
Henderson 2004, McWilliam 2008); creation of analogies and mental models (Perkins 1981); and thrive in an 
environment where unpredictability and complexity are the norm (McWilliam 2008). 

Henderson (2004) and Runco (2004) agreed that inventors possess the aforementioned characteristics and, 
in addition, display characteristics of investment of time and effort, tenacity, problem-finding vision, problem- 
solving focus, confidence in abilities, and immersion in a responsive environment. 

The Course: Innovation Technologies and Creativity in Teaching 

The “theme” for this course is Innovation Island, selected specifically to give participants (K12 teachers) a 
sense of relaxation, play, and fun. The island is home to four main learning areas: Investigation Station, 
Manufacturing Marina, Classroom Ecosystem, and Pedagogy Pavilion. Areas for learning support include a 
Welcome Center, Resources, Collaboration, and Career Planning. The course site is non-linear in nature but students 
could be guided through in a linear approach. Following is an explanation of the course learning areas. 

The Welcome Center presents some basic background knowledge including definitions of innovation, 
creativity, invention, and manufacturing and characteristics of creative, innovative learners. The course syllabus and 
FAQs are also housed here. The Resources room includes links to additional learning materials including 
presentation of different models of teaching creativity and innovation. Career Planning offers resources related to 
opportunities in manufacturing, invention, innovation and creativity. 



Investigation Station 



Investigation Station is the spot for teachers to learn problem solving through authentic challenges. 
"Problems are solved by hard work and persistence - by moving away from falsity, not moving towards the Truth. 
We learn more from failed experiments than from confirming ones" (McWilliam, 2008, p.192). 

The investigations in this section are organized into the "Think, Play, Do" innovation schema (Dodgson, 
Gann & Salter, 2005). Unlike the traditional sequences of research, development, and engineering, the Think, Play, 
Do process is characterized by highly fluid boundaries, with thinking, playing, and doing occurring at all stages of 
the innovation process. Participants are be presented with authentic challenges and guided through the problem 
solving process from problem finding through brainstorming through pattern identification problem solving (Getz & 
Lubart 1999, Adams 1972, Faste 1972, Mafarquhar 1999, Baillie 2006, Perkins 1981). 

Michalko (2006) offers a wealth of activities designed to challenge thinking habits and increase creativity. 
Activities like SCAMPER, a questioning technique to determine the strength of an idea, and mind-mapping are 
presented in this section of the course. 



Manufacturing Marina 

The goal for this section of the course is to move participants from creative thinking into the realm of 
excellent problem solvers who discover novel solutions (Henderson, 2004). Participants will be asked to involve 
themselves in the inventing process, from brainstorming, to idea generation, to rapid prototyping, to management of 
an innovation. Dodgson, Gann & Salter (2005) differentiate between innovation technologies, information 
communication technologies, and operations and manufacturing technologies. Table 1 lists the features of each of 
these types of technologies and their uses. 



Innovation Technologies 
(IvT) 


Information & Communication 
Technologies (ICT) 


Operations & Manufacturing Technologies 
(OMT) 


Function: to create 


Function: to enable 


Function: to implement 
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Search tools 


Computers and servers 


Computer Numerical Control (CNC) 


Modeling and simulation 


Open systems 


Flexible Manufacturing System (FMS) 


Visualization/virtual reality 


Bandwidth 


Computer Aided Design (CAD) 


Rapid prototyping 


Sensors 


Computer Aided Manufacturing (CAM) 




Internet/ WWW 


Computer Integrated Manufacturing (CIM) 




Wifi/3G 


Common Industrial Protocol (CIP) 




Electronic Data Interchange 





Table 1. Differentiation between generic types of technology 



Bull & Garofalo (2009) claim “the second half [of the digital revolution] involves the round trip back from 
bits to atoms to enable the creation of tangible materials based on digital designs. We will build upon their work 
with digital fabrication by setting up “Innovation Stations” for participants to use in learning. Each station will 
include a computer, auto-cad software, a color printer, and a computer-controlled die-cutting machine that shapes 
paper, cardboard, vinyl, or other similar materials. Bull (2009) offers examples of how digital fabrication is being 
used in content areas: 

• Science: Use geological and earth science resources to construct models depicting certain phenomena 

• Math: Create 3D shapes for geometry concepts 

• Elementary: use static cling inkject film to produce customized Colorforms-like sets for storyboarding 
projects in language arts or social studies instruction. 

The possibilities for enhancing student learning with digital fabrication tools are limitless and will give teachers a 
glimpse into the world of invention and manufacturing. Student motivation and ownership in conceptual learning 
will increase as well. 

Through the course, participating teachers will have the opportunity to use an “Innovation Station” and 
develop content-rich lessons for their own students. Additionally, they will develop a business plan to partner with 
one or more of the grant project’s SMMs with the goal of setting up an Innovation Station in their own classroom. 

Classroom Ecosystem 

Regarding definitions of creativity, Csikszentmihalyi (1988) noted the key question is not “what is 
creativity”, but “where is creativity.” Other researchers include social, cultural or historical dimensions in their 
conceptions of creativity as well (Amabile & Tighe, 1993; Gardner, 1993; Sternberg & Lubart, 1991; 
Csikszentmihalyi, 1996), emphasizing the importance of the environment’s contribution to the development of 
creative potential. Henderson (2004) hypothesized features of a formative environment from studies revealing the 
importance of classroom environments that support inventive behavior, while McWilliam (2008) quoted John Bruer, 
president of the James S. McDonnell Foundation, as articulating the need to value, create and sustain environments 
that are low in threat and high in challenge. 

Uden & Damiani (2007) differentiated between a natural ecosystem as “a biological community of 
interacting organisms and their physical environment” and a business ecosystem as “a network of buyers, suppliers 
and makers of related products or services and their socio-economic environment that includes institutional and 
regulatory framework” (p. 1 14). Xia (2006) notes that the term ecology is not merely a biological term but a 
scientific term that “shows concern for man’s goal, function and future, but also for the ecosystem, the diversity of 
life and the creatures’ activity environment” (p. 57). In this sense, the classroom can be described as an ecosystem, a 
systemized whole consisting of living creatures and the environment in which they exist. Xia (2006) describes 
further: 

There exists not only close correlation and interdependence among the living creatures, but also close 
correlation and interdependence between the living creatures and the environment or other non-biotic 
factors in this system; they form the life community in coexistence and cooperation. In the classroom 
ecosystem, teacher and students are the biotic factors. Both the instructor (the teacher) and the educated 
(the students) are human beings - the highest form separated from animals in the biosphere, and are 
members with responsibility and capability of giving rational answers to rational questions. The teacher and 
the students are interdependent and interactive. Either of the two is indispensable in constituting the 
curriculum and the teaching activity system. The curriculum and the teaching activity cannot function 
without specific environment. There exist close correlation and widespread interaction between the teacher, 
students and the classroom environment, and the various biotic factors in the classroom environment, (p. 

60) 
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While the teacher and students are biotic elements in a classroom ecosystem, the non-biotic elements exert 
significant influence on the learning that occurs. Non-biotic elements include: 

• Natural factors: colors of various objects, natural lighting in the classroom, temperature and noise; 

• Space and time factor: organization of classroom space and teaching time, rhythm, and efficiency; 

• Facility factor: infrastructure such as desks, chairs, platform, whiteboard; 

• Information factor: the novelty, richness, scientific-ness and vividness of information transmission and 
exchange; 

• Cultural element: the nutrient of the classroom that feeds the whole ecosystem; 

• Organization factor: different organization forms have different characteristics in influencing students in 
terms of classroom interaction, exchanges, participation and competition; 

• Emotional factor: Positive and rich emotion can promote the understanding process and the will process, 
and help in the full development of individual character; 

• Interpersonal factor: learners and teachers form a complex interpersonal relationship through multi- 
directional exchanges, which permeate and eventually affect the classroom atmosphere, classroom activity 
and the learning process; and 

• Public opinion factor: different views that arise from teaching questions or contradictions, which affect 
each other. (Xia, 2006, p. 61) 

Creativity and innovation can either thrive or die depending on the classroom ecosystem. In this course module, 
participating teachers will conduct a thorough evaluation of their current classroom ecosystem and design plans to 
optimize that ecosystem for creative, innovative teaching and learning. 

Pedagogy Pavilion 

To develop a classroom into an actual learning ecosystem, social and physical factors are not the only elements that 
need attention. Pedagogy must enter the equation as well. Bowkett (2007) suggested that teachers must have the 
capacity to make trouble for familiar thinking and the capacity to work with difference rather than move to resolve 
it. The irony of this is how different it is from conventional schools of pedagogical thought. McWilliam expanded 
that notion into a description of teachers he calls “Meddlers-in-the-Middle” (between the “Sage-on-the-Stage” and 
the “Guide-on-the-Side”). The Meddlers-in-the-Middle: 

• see themselves as designers of learning opportunities (focus on using content to create new knowledge, 
rather than memorize), 

• employ clear strategies for helping students know what to do when they don’t know what to do, 

• are good at setting up experiments that might fail, 

• help kids fail without shame, 

• are capable users of digital technologies but don’t rely on it to teach for them, 

• are pleased to learn about technology from students, 

• have a good understanding of the culture of all kids (value diversity), 

• can bring subject, technology, and culture together, and 

• invite noise, uncertainty, and argument as part of the fun of learning, (p. 108) 

She also suggests that the Meddler-in-the-Middle spends less time giving instructions and more time spent being a 
usefully ignorant team member in the thick of the action and less time being a custodial risk minimizer and more 
time spent being an experimenter, risk-taker and learner (McWilliam, 2008, p. 88). 

This section of the course is designed for participating teachers to learn and develop characteristics of the 
creative, innovative teacher and begin to employ pedagogies to foster these skills in their students. Pink (2005) 
suggests that a curriculum of creativity moves from the old to the new (Table 2). 



OLD 


NEW 


Function 


Design 


Focus 


Symphony 


Seriousness 


Play 


Argument 


Story 


Logic 


Empathy 


Accumulation 


Meaning 



Table 2. Description of changes to creativity curriculum from old to new 
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In embracing new pedagogy, it is necessary to consider assessment of student learning, whether it is consensual 
assessment (Balchin, 2006; McWilliam, 2008), self-assessment (Cowan, 2006); by the process or product (Cowan, 
2006) or purpose (Stables & Kimbell, 2007); or through conceptualization, schematization, and execution (Cowdroy 
& deGraff, 2005). 

The role of the teacher is to foster a classroom ecosystem in which new pedagogy can thrive; this begins 
with the teacher’s capacity to recognize, represent, articulate, and evaluate his or her own creativity and innovation 
(Cowan, 2006; Wisdom, 2006; Baillie, 2006; Jackson & Sinclair, 2006; McWilliam, 2008). The Innovation 
Technologies and Creativity’ in Teaching course as an element of the Innovative Partnerships for Teachers project is 
designed to immerse teachers in innovation technologies and creativity in the classroom. 
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Abstract 

What are some of the factors that affect the success (or lack of) of an institution of higher learning 
when an institution decides to move multiple, master level programs from a traditional face-to- 
face offering to distance education delivery? Funded in part by resources from the federal 
government, a university in deep South Texas attempts to increase both accessibility to 
coursework and quality of instruction in several advanced degree programs to help boost 
employment opportunities for traditionally disenfranchised students. The sheer scale of such an 
endeavor brings with it its own, embedded problems. But other complicating factors, not 
necessarily unique to this project, up the complexity level of such an attempt. Two of these 
factors are the diversity of faculty members, of whom less than 5% are from the United States 
originally, and the challenge of coordinating activities involving faculty in the United States, 
Europe and Asia. This report is intended to provide a snapshot of anticipated and unanticipated 
successes and setbacks at the end of the first year of implementation from an evaluator’s 
perspective. Lessons learned are discussed. 



Background 

Ever since the lower Rio Grande Valley (RGV) in remote South Texas became part of the United States at 
the conclusion of the Mexican- American war in 1848, the area has been ripe with possibility and fraught with 
difficulties and disappointments. Currently the poverty rate as of 2008 in this area exceeds 37% according to City- 
Data.Com. The median income of this area is below $28,000 per household as compared to a national average that 
exceeds $38,000 per household (US Census, 2008). Median age for the lower Rio Grande Valley averages below 24 
years as compared to a national average that exceeds 34 years (ibid). 

Education is the key to tapping into the vast heretofore undeveloped human potential of the region. 
However, taken as a group, Hispanics account for less than 6 % of recipients of master level degrees (National 
Center for Educational Statistics, 2007). When one disaggregates the data to look specifically at Mexican- 
Americans, and further look at degree recipients in areas such as advanced mathematics and computer science, this 
6% significantly diminishes even further. 



Introduction 

With the help of some federal resources, a local university has decided to move three of its master level 
programs from traditional, face-to-face delivery to initially hybrid delivery, to transition to fully online delivery. 
This decision to move master level programs initially to a hybrid format for delivery and ultimately to a completely 
online delivery format was made in large part from a needs analysis that showed that the overwhelming majority of 
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those students who already held degrees at the undergraduate level were employed in full-time professions that 
made their seeking an advanced degree problematic at best. 

There are currently three master level degree programs involved in making this transition from face-to-face 
to online delivery. Those programs are: a master degree in mathematics, a master degree in computer science, and a 
master degree in interpretation and translation (Spanish to English, English to Spanish). While each of these three 
programs has its own set of unique characteristics in terms of content and traditional forms of instruction, it became 
apparent that certain design factors and implementation factors permeated all three of these degrees. Specifically 
those factors are as follows (phrased and interrogative form). 

1 . How does one transition faculty members from face-to-face to online instruction? 

2. What types of instructional tools are effective (and ineffective) when trying to reach end users who, for the 
most part, have English as their second language? 

3. And finally, what is the impact on the institution in the tradeoff between the strain on existing resources 
and the gain in student enrollment at the master level? 

A further, complicating variable was the ethnicity of the faculty members themselves. Of the more than 25 
faculty members involved in this project, only one is native to the United States. One other faculty member is from 
Australia, and even though English is his first language, his accent is considered by most local residents to be that of 
a foreign language. Another complicating variable is the distances involved. In one of the degrees, the degree in 
translation and interpretation, the majority of the faculty work in institutions outside the United States, specifically 
Ireland and Spain. 



Literature Review 

When hybrid course offerings are compared to traditional, face-to-face course offerings, the effectiveness 
of the hybrid courses is no longer in doubt; in fact, there is significant research that shows hybrid courses to be 
superior to face-to-face courses (Dziuban, Hartman, and Moskal, 2004; Johnson ,2000; Moor, 2004; Jones, 2009, 
Vaughan, 2007). However, there is still significant discussion as to what form courses should take as they migrate 
from face-to-face to hybrid and into completely online delivery. When developers step back and look at program 
development as compared to single course development, prevailing logic is that all courses in a degree program 
should be designed with a common scheme to both the "look" of the courses in the program and to the methods of 
instructional delivery. While students in a traditional program can transition easily between different instructors' 
methods for delivering instruction and use of various types of tools of instruction, students in hybrid and distance 
education courses can become much more easily disoriented and subsequently confused by a variety of instructors' 
delivery styles and instructional approaches (Amrein-Beardsley, 2007; Boyle, 2003). This need for consistency 
among courses in a program creates the need for each faculty member in that program to not only adopt similar 
course structure but also similar instructional styles. But having all faculty members and a program on the "same 
page" in terms of instructional look and feel as well as instructional methodologies is no simple task. Anecdotally 
one can say this is akin to trying to herd cats, especially when the "cats" all possess advanced degrees and in many 
cases at this particular University, numerous accolades in their respective academic areas. Effective training 
becomes more critical than ever. Coordination of such training, particularly when it involves intercontinental 
scheduling, becomes a significant challenge. 

Existing research supports the idea that moving into a hybrid or online format should not displace lecture 
format altogether (Boyle, 2003; Khine, 2003; O'Toole, 2003). But inclusion of lecture format into online delivery 
raises two, practical problems: the delivery quality of the lecture files to remote users, and the auditory accessibility 
to those remote users of lectures delivered by faculty with discernible, foreign accents. It is a common, if misplaced, 
complaint among American undergraduates that having a faculty member lecture to those undergraduates when the 
faculty member has a noticeable foreign accent makes attentiveness to that lecture content difficult at best (Kavas & 
Kavas, 2008). The nature of the content in advanced mathematics, computer science, and linguistics courses makes 
audio discern ability all the more important in online materials in those materials are delivered in lecture format. 
There are tools that enable listeners to easily follow faculty members who present their lectures in an online format. 
But if standardization in course content and delivery is recommended as earlier noted, then getting faculty "buy in" 
to using lecture delivery that is enhanced by speech recognition tools, etc. becomes all the more urgent and yet 
problematic among a highly diverse faculty. 

In advanced degree programs such as computer science and mathematics that inherently entail abstract 
symbolic manipulation, the need for students to interact in real time (traditionally in labs or in study groups) must be 
met if those students are going to succeed in meeting the demands of challenging coursework. The same needs of 
students for real-time interaction literally speak for themselves in a translation and interpretation degree program. 
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The tools for virtual online meetings are rapidly becoming ubiquitous and students are more and more expecting the 
use of such tools in hybrid and distance education courses. One only need think of the Super Bowl in 2010 and the 
commercial advertising GoLive meetings during that televised event as an example of the ever-increasing presence 
of online conference management systems. While these types of conference facilitation tools can provide a 
significant amount of value added to individual courses in a program collectively, getting faculty to systematically 
use these tools, particularly when those faculty are not native speakers of English and English will be the 
predominant language of instruction in audio environment, can be an extremely difficult undertaking. 

The current project of moving three master level degree programs initially into a hybrid and then into an 
online environment is currently completing its first semester of implementation. The process for this migration at 
this point is as follows. Two courses will be piloted in each of the three master level programs. These two courses in 
each program are initially designed to use the same tools and the same form of content delivery. Training is being 
provided to respective faculty members by having University staff travel to foreign universities when necessary and 
deliver training in a face-to-face environment. However, when the project scales up as it will have to do to meet the 
needs of a total of 36 course offerings in the three respective degree programs, effective faculty training will require 
the development of modules that faculty can access in a "just in time" fashion. The successes and "non-successes" of 
various types of training sessions and faculty members’ usage of various types of tools is currently being 
documented so that subsequent modifications can be made when necessary. 

The ethnicity of the faculty members for the six pilot courses were as follows: two faculty members from 
the People's Republic of China; one faculty member from South Korea; one faculty member from Germany; and two 
faculty members from Spain. Four of the faculty members involved in the programs were located on campus at the 
University of South Texas. Of the two remaining faculty members, one faculty members delivered a course from 
Spain and the other faculty member at delivered a course from Ireland. Faculty members involved in the 
mathematics degree and in the computer science degree all reside within driving distance of the University. Faculty 
members involved in the translation degree reside in Ireland and Spain for the most part, but routinely moved to 
other countries in Europe and North America. Training for faculty members in all three programs was absolutely 
essential since the large majority of the faculty members involved had either designed and online course north taught 
in online course. The mathematics and the translation programs benefited significantly, however, from the inclusion 
of one faculty member in each program who had considerable experience in designing, developing, implementing 
and evaluating online courses. The computer science program, while having no faculty members experienced in any 
aspect of distance education, had a co-principal investigator who likewise had years of experience in program 
development and implementation in online education. This also indicated how the distance factor imposed to the 
transition process. 

To sum up, this project of moving a total of 36 courses in each master program from its traditional offering 
into a hybrid offering and then into a completely online offering is in its first year of implementation. The project is 
highly ambitious not only in its scope but in its attempt to develop and deliver the highest quality instruction 
possible for an intended audience of traditionally disenfranchised Mexican-Americans. Participating faculty in this 
project, with one exception, come from countries other than the United States. These faculty members bring a 
wealth of experience, both professional and personal, to the project. At the same time these faculty members come 
from cultures steeped in tradition, especially the understanding of what traditions of education entail. Documenting 
and reporting upon efforts in helping these faculty members make the transition from traditional to virtual 
instruction is both timely and appropriate as universities around the world continue to make the transition to online 
instructional domains. 



Method 

Evaluation of the first iteration of the project (the course offerings for the first semester) looked at the 
following factors: training for faculty members, course design, course delivery, and student outcomes. Data were 
collected from the following sources: attendance at, and monitoring of, faculty training sessions; accessibility to all 
stages of individual course development for the evaluator, from initial planning meetings through “course launch” 
on the first day of the semester; evaluator’s inclusion in the courses; evaluator’s access to results of students’ 
assessments in individual courses, final grades, and students’ evaluations of courses and instructors. Data consisted 
of direct observation of training sessions, viewing video archives of training sessions that occurred abroad, inclusion 
of electronic conversations between individual faculty members and their assigned instructional designers as courses 
were negotiated and developed, inclusion in various forms of communications common to online and hybrid courses 
(e.g., announcements, broadcast e-mails, discussion boards, archived videos). 
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Prior to the initial course offering in the three programs, faculty members who were offering the initial 
programs’ courses attended at least 10 meetings to familiarize themselves with the multiplicity of issues involved in 
moving courses from a traditional format into a hybrid or completely online format. These were not training 
sessions where participants would learn how to use discussion boards for example. These meetings were more of a 
strategic bent, where issues ranging from student recruitment to standardize course design were raised and 
discussed. Each meeting would have a very specific agenda. Notes would be taken. Action items were proposed 
and previous activities were evaluated. The evaluator had full access to all these meetings and took notes 
accordingly. Those meetings involving the translation program were video-archived since these meetings could not 
occur on the University campus, with two exceptions. 

The two instructional designers were assigned to these three programs. The instructional designers 
included the evaluator in their correspondences with faculty members when those correspondences involved in the 
issues related to design courses and instructional strategies. At the beginning of the semester, the evaluator was 
included in each course and was allowed full access to all components of each course. Individual instructors would 
not send specific correspondences between a student and an instructor to the evaluator, as is in keeping with the 
instructor student confidentiality. However, one faculty member in particular would routinely summarize larger 
issues that an individual interaction with the student represented and would send that synopsis of the evaluator. 
These “abridged e-mails” turned out to be quite a boon for uncovering institutional shortcomings in the institutional 
approach to distance education. 

The evaluator was allowed to peruse assessments (grades) and individual learning modules as long as the 
confidentiality of the student was not violated. Therefore, the evaluator was given access to final grades and student 
evaluations. 



Findings 

Each program had a series of initial meetings of which the purpose was to address strategic issues. Since 
there were a relatively small number of faculty involved in these initial course offerings, scheduling these meetings 
was not excessively difficult, even for those faculty members who reside abroad. These meetings appeared highly 
beneficial in raising problematic issues and addressing those issues before they became systemic problems. Those 
issues involved matters routinely encountered in programs moving from a traditional format to a hybrid format or 
completely online format. Faculty members in the pilot course offerings in all three programs were included in the 
discussions of the benefits of including a standardized instructional approach and course structure for the initial 
courses in each program and for subsequent courses that would be developed in each of those programs. The 
predictable resistance to the perception that faculty freedom was somehow being stripped away, or at least 
homogenized, was addressed in a satisfactory fashion and all participants in the pilot program saw the wisdom, and 
more importantly, the benefit to the students, for a similar “look and feel” in each course. 

Training became less complicated since the initial courses in each program would have the same course 
structure and embedded instructional strategies. Each program benefited significantly from the presence of highly 
experienced individuals in distance education (two faculty members and one co-principal investigator). Those 
individuals could explain to their peer faculty members the wisdom of using a standardized approach that still 
allowed individual instructors to exercise their academic and scholastic freedoms. Training sessions could then be 
delivered “lockstep” in which a specific topic germane to each program could be explored in depth, rather than 
having to work with each individual faculty member and address specific issues to whatever design decisions that 
faculty member may be making. On two occasions, however, the lead faculty member in the translation program 
had to go abroad, to Spain and to Ireland, to help deliver training sessions in person. The course management 
system being used, Blackboard, proved problematic to faculty members who could not be in the same room as their 
trainers, at least for initial sessions. On one of these trips, the lead faculty member had to take an instructional 
designer, incurring an extra cost, but ensuring successful training. 

There were two significant barriers to success in these initial course offerings, each obstacle specific to a 
particular program. The first obstacle turned out to be a language barrier. Those professors with a foreign accent 
have been educated in their advanced degrees in the United States. In small meetings, or in one-to-one 
conversations, their version of American English is easy to understand. However, their accents often became 
incomprehensible when they would present lectures and have those lectures turned into video archives for their 
hybrid classes. The evaluator never sat in on a particular, traditional face-to-face class that these professors were 
offering in which the video archives were recorded. But the evaluator has spent numerous hours in person with 
these professors on some of the archives. There was a significant difference in the perceived quality of their usage 
of American English between their face-to-face conversations and the video archives. This difference in quality 



294 




substantially reduced the instructional capacity of the archive lectures. A solution was developed and implemented 
towards the end of the pilot semester in which the archive lectures were still available for student access. But 
modules were developed to complement those archive lectures. Each module contained all the elements of sound 
instructional design, interactive quizzes, Flash animations when appropriate, but most importantly, uses a Native 
American English voiceover instead of the professor's voice. The professor would write a script to complement 
contents of the module and the narrator would read from that script. This approach to design of instruction was 
more time-consuming and resource intensive, but significant in improved students’ satisfaction with audio quality, 
and assisting those students in mastering instructional objectives embedded in each module. 

Predictably, faculty in the translation program had no issues regarding language usage or accents, other 
than an oddity found in common language usage in different cultures. Professors had to be sensitive to specific 
word usage. For instance, the meaning of specific Spanish word in Barcelona may not have the same meaning in 
Mexico City. The issues that plague the translation program were institutional. As mentioned, the University is 
located in far South Texas. The institution has developed a provincial attitude due to its remoteness. Issues, such as 
student applications for admissions, financial aid, and course registration, have all been traditionally done in a face- 
to-face environment. Moving a course from a face-to-face delivery mode to an online delivery mode involves one 
set of issues. Moving an institution into a position where it can support multiple online program offerings involves a 
completely different set of issues. Because the computer science program and the mathematics program were 
focusing upon recruiting students within driving distance of the campus, institutional support for distance education 
students did not significantly impact those programs, at least during the pilot course offerings. But, students 
applying from other parts of the state or other parts of the country or from other countries encountered an unusual 
mix of aggravation. Some of the types of difficulties that students and professors in Spain and Ireland encountered 
are as follows. 

In the application for admission process, students are required to supply student visas if they are from a 
foreign country. No provision had been made for students who are going to take online degrees and never set foot 
on campus. Applying for financial aid required face-to-face counseling, highly problematic for students located 
some hundred miles away. The institution did not appeared prepared for distance education professors who reside in 
different countries in terms of expediting work visas or direct deposit to bank accounts. 

Conclusion 

Ideally, an institution will have 2 to 4 years to prepare for moving a master level program from a traditional 
delivery methodology into an online format. However, most institutions simply don't have that luxury of affording 
the requisite amount of time for in-depth planning. Faculty members, typically in areas such as foreign languages, 
mathematics and computer science are more and more representing a global community rather than a regional 
population, even at smaller universities in remote areas such as South Texas. Instructional issues, delivery media, 
authentic assessment are only some of the issues involved in transitioning a program from a traditional format to a 
completely online format. One can think of a tripod when conceptualizing the needs of a distance education 
program. One leg represents the instructional component, one leg represents the technological demand both for 
delivering and receiving instruction, and the third leg represents the administrative components of any institute of 
higher education that service faculty and students, and must be prepared to service faculty and students at a distance. 
While distance education has been mainstream in United States universities for at least 10 years and most issues 
have been effectively addressed, offering online advanced degree programs to local, national and international 
students, from faculty members who represent a microcosm of the United Nations is still new territory. From the 
evaluator’s perspective, each program needs a champion in the initial course offerings. In this context, each faculty 
member involved needs to be a team player first, an award-winning academic second. If resources permit, all 
components of the institution that support academic programs need to have a designated contact person who can be 
reached at and have a solution to a problem within 24 hours. 
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Abstract 

An Immersive Security Education Environment is a serious game running on the Second Life environment. 
The purpose of the game is to allow players to experience various information security mechanisms and attacks 
when they do not have appropriate security protection. In this game, teams of students will play against each other 
trying to gain the highest scores in all three major areas- Money, Information Security, and Product Diversity. 

Teams operate their own online stores selling computer equipments that they have to purchase them online. Teams 
have to engage in a scavenger hunt activity to find Second Life objects and their associated quizzes to earn the 
rights to use various information security mechanisms. In addition, each team has to set up appropriate security 
mechanisms for their online stores to avoid being cyber-attacked. Playing this game will provide a unique learning 
experience that players cannot find from the lesson in the classroom or from the textbook. 

Introduction 

Information security knowledge and skills are important for all Internet users. Most of us use the Internet 
to share personal and other sensitive information without thinking much about security. For example, when you 
purchase products online, do you always check that website’s privacy policy? Do you check if a seller’s website 
implements any security features to protect your information? It is likely that the answers to these two questions are 
“maybe” or “rarely”. At Penn State University, we developed a novel experiential learning module offer a course 
that provides basic information security education. During the course, instructors will discuss key information 
security concepts and security mechanisms such as port blocking, firewalls, and cryptography through traditional 
lectures. However, there is little evidence that the lecture format is an effective learning approach for information 
security topics. One of the challenges for the instructor is to provide real-life experiences to the students. 

Essentially, this is the underlying motivation for the development of this game. Playing a game is an activity of 
improving skills in order to overcome challenges from the rules of the game. Therefore, game playing is one of 
fundamental learning experiences (Juul, 2005). The game was designed and developed by a group of Information 
Sciences and Technology (1ST) instructors and students. The game was used in the classroom for a few years and 
received positive responses from students. Furthermore, the students’ learning performance has been increased in 
terms of test scores and their knowledge. In this paper, the structure of the game and the game design will be 
presented. 



Rationale 

All Internet users need to have basic information security education. However, information security 
education primarily focuses on the technical aspect of information security. In addition, most of the contents are 
delivered as abstract concepts without much attention on their meaning in real-life contexts. Therefore, these 
learning experiences have very little impact on users’ information security behaviors. To address these limitations, 
we will discuss the use of a simulation game using an integrated Web and Second Life environment to educate 
users about safe online behaviors. Simulations can enhance safely, provide experiences not readily available in 
reality, modify time frames, make rare events more common, control the complexity of the learning situation for 
instructional benefit, and save money (Alessi & Trollip, 2001). In this simulation, students are given a scenario in 
which they have to compete with others to make their online stores and private information as secured as possible. 
The game also provides an opportunity to measure their understanding of information security concepts. In 
addition, fun is the most important factor, which will motivate students to learn more effectively. 
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Structure of the Game 



Scenario: You and your team members have been put in charge of opening a new virtual store in Second 
Life. Your main missions are 

• Build a secure IT infrastructure for your new store 

• Stock the store with goods purchased from trustworthy e-commerce Websites 

• Sell products to customers 

• Securely protect information assets (e.g., password, credit card information, customer information) 
Challenges: 

• All tasks need to be accomplished within a limited budget 

• Your team competes against other teams 

• Random attacks will be launched against common information security vulnerabilities 

• No single best strategy to protect information assets 



How to play the game: 

• Each team will receive the team assets: 
o Password protected team site 
o Credit card with $5,000 (see Figure 1) 
o Attractive avatar 
o gmail account (see Figure 2) 
o Second Life store space (Figure 3) 




Alpha's Profile Page 



Your Credit Card Info About Avatar Alpha 



Figure 1: Credit Card 




Home Tutorials Activities Ecommerce Avatar Team Info Logout 



Team Alpha Information Page 



I Gmail Account 



E-Mail Information 



Instructions and Guidelines 

• Go to the user login at aooale.com 

• Log in and change your password: 

Avatar Name: alpha.actor@gmail.com 
Password: secondlife 

• Change your password with the provided form below. 

• An E-mail with the new password will automatically be sent to all team members. 

• NOTE: Points will be deducted if you do not update the form below to match your e-mail's new password! 



New Password: 
Repeat Password: 




Figure 2: gmail account 



298 







Figure 3: Second Life store space 



What should a player do in the game? 

• The player will perform router and firewall configuration of their online store. 

• The player will set up security countermeasures such as port blocking, CAPTCHA, out-of-band, security 
images, and clipping for the online store. 

• The player will protect their online store from security attacks such as spam, password cracking, website 
defacement, zero-day attack, and port scanning. 

• The player will also practice safe online behaviors such as careful choices of stores to purchase products, 
password choices, privacy policy, and access controls. 

Followings are the tasks that the player performs: 

Taskl: The players have to purchase IT equipments for their store. There are four stores that sell IT equipment. 

However, ach store has different levels of security, (see Figure 4) 
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Home 



Seed Stores 



SRA 221 - An g sana 



SRA 221 - Jung woo 





Team Upsilon 




Home 
Project 
Team Page 



Team Zeta 's E-commerce Site 




Privacy Policy 

Our Policy: 

570-000-0000 

Zeta.Zaytsev@Gmal.com 

http://isee-istpsu.edu/Zeta/ 

Beta Burch 
Istanla 2 Second Life 
State College 
Pennsylvania 
16802 

United States of America 
Zeta 

Security Features this site offers: 

Clipping: Ye. 

Captdia: Ye. 



Out Of Band: No 




Welcome! 



Team Zeta 's 




Team Zeta 



Figure 4: Purchasing IT equipments 




After the products are received, the players have to set up the IT equipment and configure the router, (see Figure 5) 




Figure 5: Set Router Configuration 
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Then, the players have to set up their online store as well as engage in a scavenger hunt (See Figure 6) in order to 
gain security features for their online store. The security features include CAPTCF1A, Security image, P3P, Out of 
band, and Clipping. They have to take a quiz for each security feature and get at least 80% of the test in order to ear 
the right to use a security feature, (see Figure 7) 




Figure 6: Scavenger hunt 



1 ) Scavenger hunt: 
CAPTCHA 
Security image 
P3P 

Out of band 
Clipping 

1) Take quizzes 

Criteria: 

80% passing score 




Welcome to the OutOfBand Security Quiz! 



Object with embecded 

out of ban d quiz _ 

1 



Figure 7: Out of Band Quiz sample 

After the store is set, the players will start buying and selling products, (see Figure 8) 
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Home 


Tutorials 


Activities 


Ecommerce 


Avatar 


Team Info 


Logout 



About your Team | Layout Options | Store Name 



Sales I 



Items you have Sold. 



|Product Name 


|Sold To 


||Date 


(Quantity 


|Pr1ce 


|Partide Emitter 


|Beta 


|November 17, 2009, 1:53 pm 


|i 


$100 


|sofa 


Beta 


|November 17, 2009, 1:53 pm 




$300 


|Mahogany Table 


|Beta 


(November 17, 2009, 1:53 pm 


|i 


$250 


[Total Sales 


_l 


II 


II 


$650 



Figure 8: Buy and Sell Products 



At this point, the instructor will act as an attacker and start launching security attacks to student teams, (see Figure 

9) 



Home Teams » Classes » 



Second Life Module Manager 



Faculty Attacks » 



THE PENNSYLVANIA STATE UNIVERSITY 
Bank » Messageboards Scoring Products Log 



Launch Attack 


View 

File: 


Date: 


Class 

ID: 


Class Name: 


Attack Type: 


Attack: | TeamSIte Password * 




Link 


October 27, 2009, 4:21 pm 


2 


SRA 221 - 
Jung woo 


Router Configuration 


Captcha (Insert Fake Users) 
jRouter Configuration^ ^ 


Link 


November 16, 2009, 5:33 
pm 


1 


SRA 221 - 
Angsana 


Captcha (Insert Fake 
Users) 


Link 


October 19, 2009, 1:25 pm 


1 


SRA 221 - 
Angsana 


Router Password 




Link 


November 16, 2009, 5:33 
pm 


1 


SRA 221 - 
Angsana 


Captcha (Insert Fake 
Users) 


( Launch Attack 




Link 


October 22, 2009, 2:20 pm 


2 


SRA 221 - 
Jungwoo 


Router Password 


O Copyright Penn State University. All Rights Reserved 


Link 


October 21, 2009, 3:26 pm 


1 


SRA 221 - 
Angsana 


Firmware Upgrade - 
Attack 




Link 


October 21, 2009, 2:55 pm 


1 


SRA 221 - 
Angsana 


Router Configuration 


Link 


October 21, 2009, 3:33 pm 


1 


SRA 221 - 
Angsana 


Router Password 


Link 


November 16, 2009, 5:36 
pm 


1 


SRA 221 - 
Angsana 


Captcha (Insert Fake 
Users) 


Link 


November 17, 2009, 1:53 
pm 


1 


SRA 221 - 
Angsana 


TeamSite Password 


Link 


December 11, 2009, 12:16 
pm 


1 


SRA 221 - 
Angsana 


Team Site Password 


Link 


October 21, 2009, 3:08 pm 


1 


SRA 221 - 
Angsana 


Captcha (Insert Fake 
Users) 



Figure 9: Security attacks 



If student teams fail an attack, they have to contact a consultant to resolve the consequences of an attack, that the 
consultant will charge $500 fee to solve each failure, (see Figure 10) 
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Figure 10: Attack Recovery 



At the end of the game, every team will receive a score card explaining the detailed of the breakdown of their 
information security scores, (see Table 1 and 2) 



TeamName: Alpha 

Money: 2950 

Total number of products: 8 
Total Score: 44.375 % 



Site Score 


Router Score 


Survival Score 




> Team PW strength: 0% 
« Clipping: 0% 

« Captcha: 0% 

• Security Image: 15% 

» OutOfBand: 15% 


■ Router Password Strength: 12.5% 

• Firewall: 0% 

• Firmware: 25% 

• Port configuration: 0% 


• Times Down: l(-5%) 

• Times Defaced: 2(- 10%) 

» Number of Fake Users: (-0%) 


{Shopping Score 

{Beta's Site Score: 30% 
|Upsilon's Site Score: 0% 
{Zeta's Site Score: 0% 

[Total Shopping Score: 10% 


• P3P quiz: 0% 

. P3P Policy: 15% 






Site Score: 45% 


• Router Score: 37.5% 


Survial Score: 85% 





Table 1: Team Alpha sample score sheet 



TeamName: Beta 

Money: 4250 

Total number of products: 10 
Total Score: 50.625 % 




Table 2 Team Beta sample score sheet 
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Evaluation Criteria 



Each team will be evaluated based on money balance, information security score, and diversity of 
products in their store. They can view their score and money balance on the score board in the middle of the island 
during the game, (see Figure 11) 




Figure 11: Scoreboard 



At the end of the game, the winner will be announced and rewarded with bonus points. 



Conclusion 

The Immersive Security Education Environment (I-SEE) is a project that uses an integrated Web and 
Second Life environment to educate users about various information security mechanisms, safe online behaviors, 
and the consequences of security attacks. The collaborative team structure enables learners to engage in enjoyable 
and competitive learning environment. In the future, the in-depth behaviors of the learners will be investigated and 
the game design will be tested with broader types of audiences. 
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Abstract 

In today’s increasingly technological world, students are requiring more and more specialized computer 
software to complete their studies and to obtain real world, hands-on experience. While this is a challenge for on- 
campus, traditional computer labs, it is even more difficult for distance education. The University of Toledo has 
built VLab, a collection of highly customized virtual machines available to students around the clock, from 
anywhere in the world. 

Introduction 

The term “distance education” often provokes thoughts of content delivery, such as text, video and audio 
materials, to students at a remote location. Technologies such as e-mail, message boards, wikis and content 
management systems are enabling educators and learners to share information more rapidly than using traditional 
means such as correspondence. These technologies are all vital to educators in today’s high speed, competitive and 
“global” world. Yet, there are additional aspects of distance education that these technologies simply do not address. 
Consider the following excerpt from Wikipedia on the definition of distance learning: 

Distance education, or distance learning, is a field of education that focuses on the pedagogy’, technology’, and 
instructional system designs that aim to deliver education to students who are not physically "on site" in a 
traditional classroom or campus. It has been described as "a process to create and provide access to learning when 
the source of information and the learners are separated by time and distance, or both. " In other words, distance 
learning is the process of creating an educational experience of equal qualitative value for the learner to best suit 
their needs outside the classroom. Distance education courses that require a physical on-site presence for any 
reason including the taking of examinations is considered to be a hybrid or blended course of study, (Honeyman and 
Miller, 1993). ” 

With this definition in mind, consider for a moment the tools students require to be successful in their 
“educational experience”. Surely, regardless the field of study, most students will require the use of computers and, 
more specifically, application software. At the University of Toledo, students rely on Maple for learning 
mathematics, SAS and SPSS to analyze statistics, ArcGIS when studying geography, Microsoft Office for a variety 
of purposes; and the list goes on and on. A typical university today might utilize dozens or even hundreds of 
applications, many of which are highly specialized for specific fields of study, and expect students to demonstrate 
some level of competency in using these tools. When we consider this aspect of the educational experience, we find 
a challenge in providing effective distance education. If what we are trying to deliver to students is not simply 
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content, but content coupled with the hands-on application using computer software, how do we do it? This question 
is at the heart of this paper. 

Traditionally we’ve had two options to address this issue: The first is to have students purchase the 
software and install it on their own personal computers. The challenge with this method is that software is often 
prohibitively expensive and might serve little purpose to a student once a course is complete. In addition, there will 
undoubtedly be technical issues with system requirements and configuration that will affect the learning experience. 
The second option is to provide content remotely and require students to come on-site to use a computer lab. When 
we consider the definition of distance learning above, this option really shifts us away from distance learning to a 
hybrid or blended model. In this paper we are proposing a third option, one that attempts to address the 
shortcomings of the others, and provide a solution for this increasingly important component to distance education. 

The University of Toledo 

The University of Toledo consists of more than 20,000 students studying across multiple disciplines as well 
as a 300-bed teaching hospital. To address the needs discussed in the last section, we’ve essentially built a private 
cloud using an assortment of VM ware technologies. This will be discussed further in the following section titled 
“Technology”. We call our cloud “VLab” ( http://www.utoledo.edu/it/vlab ), and it is essentially a collection of 
virtual machines tailored to the specific configurations of each of our colleges. The primary goal of VLab is to tear 
down the time restrictions and physical boundaries of existing physical computer labs. This collection of virtual lab 
computers can be accessed via the internet from anywhere in the world, around the clock, using a variety of end- 
point devices such as Microsoft Windows PCs, Macs, thin clients, and mobile phones. Once connected through a 
secure, encrypted tunnel, students are presented with their own virtual machine, fully configured with the university- 
owned software they need to be successful. Students taking DL courses are now able to obtain media from 
Blackboard, our content management system, and then complete assignments requiring the use of specialized 
applications using a virtual lab; all without setting foot on campus. 

Today, we have approximately 1,000 virtual machines running nearly three dozen unique configurations for 
individual colleges and specific courses. While remote access for students to our VLab has been the primary focus, 
we’ve found a variety of other uses for this technology. Our dormitory lab computers, for example, have all been 
replaced with Wyse thin client devices which provide access to the VLab virtual machines. This configuration 
provides our students with a unified computing experience whether they’re on or off campus. It has also saved 
individual departments on hardware acquisition and support costs, in addition to reducing energy consumption by 
approximately 50% (Coyne, 2008) in these areas. A similar configuration is being used to deliver a more stable, 
secure computing environment for nearly half of the UT medical center hospital. At the time of this writing there are 
more than 600 thin client devices throughout the UT enterprise that are being used to connect to the private cloud 
infrastructure. 

Technology 
Private Cloud 

The term “private cloud” has been defined several ways. Consider the following examples: 

“Cloud computing is Internet-based computing, whereby shared resources, software, and information are provided 
to computers and other devices on demand, like the electricity grid (Wikimedia, 2010).” 

“Private cloud (also called internal cloud or corporate cloud) is a marketing term for a proprietary > computing 
architecture that provides hosted services to a limited number of people behind a firewall (TechTarget, 2009). ” 

VLab is all of this. As you’ll see in more detail below, there is a great deal of technology, both hardware 
and software, driving the virtual lab environment. But none of this is seen by student. They simply browse to a web 
page, begin accessing compute resources as needed, and then close the page when they’re done, freeing up those 
resources for the next person. 

Virtual Machine 

Before we go any further it is important to understand what a virtual machine is and how it differs from 
traditional computers. Wikipedia defines virtual machines as either system virtual machines or process virtual 
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machines (Wikimedia Foundations, Inc., 2010). In this document I will only be discussing system virtual machines 
which, from this point forward, will be referred to as either “virtual machines” or simply “VMs”. To understand 
what virtual machines are, let’s first take a brief look at how traditional, physical computers work. 

There are two primary components to a computer: hardware and software. Hardware is anything that you 
can touch such as the, keyboard, mouse, or hard drive. Software refers to any program that makes the computer do 
something. Today, most computers are first loaded with operating system software such as Microsoft Windows or 
Apple OS X. The operating system controls interaction between the various hardware components (such as a video 
card and memory) and applications that are installed (such as a word processor or statistical package). Below is a 
simple visual showing the relationship between hardware, operating system software, and application software. 



Figure 1 - Logical layers of a traditional computer 




With traditional computers, we see one physical machine (hardware) running one operating system and one 
or more applications. With a virtual machine however, we separate the hardware, add a virtualization layer (also 
known as a hypervisor) and gain the ability to run multiple operating system instances on a single piece of hardware. 
We are effectively turning one physical machine in to several computers. In the virtualization world, the new model 
looks like this: 

Now that we’ve established our definition of a virtual machine, let’s look at some of the technology we use 
to build and manage them for VLab. 

VM ware View 

VMware View is at the core of VLab. This product is used for provisioning, managing, and brokering 
virtual machines. The importance of these concepts is described below. View has a significant advantage over 
competing products such as Microsoft Terminal Services and Citrix because of its unique approach to virtualization. 

Rather than taking a single operating system and application configuration, slicing it up, and sharing 
“sessions” between multiple users, View gives each user his/her own virtual machine. In fact, it presents the ability 
to provide a brand new virtual machine to each user every time they access the virtual lab. This is a fully automated 
process, relieving pressure from the IT support staff. This configuration is one we rely on heavily in order to ensure 
the best possible experience for our students. 

Each virtual machine is complete with an operating system, a set of applications, and a custom 
configuration to match the needs of the college or specific course. The hardware (processor, memory, and storage) 
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resources are shared between active virtual machines and are allocated dynamically. The ability to be infinitely 
flexible in configurations while managing the entire environment with a single tool is what made View our product 
of choice. 



Figure 2 - Logical layers of a virtual machine 




Provisioning 

Provisioning refers to the process of creating and destroying virtual machines. VMware View allows an IT 
administrator to rapidly create one or hundreds of virtual machines in a matter of minutes. View relies on templates 
to create collections of virtual machines called pools. Each pool can use a unique template giving the VMs it 
contains the specific attributes required by the users who will be entitled to use them. These attributes include but 
are not limited to operating system, application suite, and the amount of hardware resources that can be allocated. 
Once a pool is created, provisioning is a fully automated process. Virtual machines can be automatically destroyed 
when a user is done with them and new VMs can be generated in their place. 

Managing 

Managing the lifecycle of computers can be a daunting task. A significant amount of IT resources are spent 
reinstalling operating system and application software in traditional computer labs. Updates often require planned 
outages and a great deal of redundant human effort. The following is a quote from the Gartner group about total cost 
of ownership of a traditional PC: “ For a large company, the cost of purchasing a desktop PC may be only $1,200, 
but, kept for four years, the total cost of ownership (TCO) could be as much as $5,867 per year, according to 
(Gartner, Inc., 2010) ” VMware View provides a single management interface to manage all virtual machine pools. 
Templates can be updated offline and used to rebuild pools of VMs with minimal effort and no user downtime. This 
feature frees IT resources to work on higher level, strategic initiatives, dramatically reduces TCO of individual 
machines, and improves student experience. 
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Brokering 

By definition, the virtual machines in our private cloud are all housed behind the university firewall. This 
protects the devices from malicious software and unauthorized access. The View Connection Broker provides a 
means for authenticating the user against Microsoft Active Directory, verifying their entitlement(s), and managing 
the session traffic between their end-point device and the virtual machine. 

VM ware ESX 

VMware ESX is a proprietary operating system that creates the hypervisor layer and makes virtualization 
on the x86 server architecture possible. The University of Toledo has virtualized nearly fifty percent of its server 
infrastructure over the last five years using this technology. At the time of this writing we have just over 260 
virtualized servers running on approximately thirty-five physical devices and this number is growing all the time. 

VMware View builds on the existing ESX infrastructure to build virtual machines. Because we already had 
an existing ESX infrastructure for server virtualization, it was a relatively simple step to start vitualizing the desktop 
and to build the VLab. 

Conclusion 

While opinions certainly differ on the role distance education will play in the future of education, 1 think it 
is safe to say that its prevalence will continue to increase as it has in recent years. Educators will need to embrace 
technology that empowers students to be efficient and effective learners outside of the classroom as well as in. 
Building a private cloud infrastructure on VMware technology has proved to be an effective means of doing just 
this. While there are technical limitations to every solution, we are finding that one of our biggest challenges is 
keeping up with the demand for more infrastructure and more virtual machines. As an IT professional, this is the 
best kind of problem to have. 

Figure 3 below provides an overview of the components required to build VLab. It also illustrates the 
process flow for both provisioning virtual machines (in red) and connecting to them (in blue). 
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Figure 3 - VLab Architecture Design 



VM Pools 
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Figure 4 - Power Study Results 



The following data are the result of a UT power study conducted by Jeffrey Coyne in Information 
Technology with the help of UT electricians. The original data have been modified minimally for easier 
display. All measured wattage has been rounded off to the nearest whole number. All costs have been 
rounded to the nearest cent. Some columns that were not applicable to the topic at hand were removed. 







Low 


High 


Screen 


Monitor 


Device Type 


Power Off 


Utilization 


Utilization 


Saver 


Off 


Wyse Terminal 


2 


13 


13 


13.000 


13 


Dell Optiplex 755 
Dell Optiplex 755 Energy 


7 


49 


84 


53.000 


48 


Star 


7 


41 


76 


46.000 


41 
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Scenario Two: 24/7/365 student lab OR hospital 



In both 24/7/365 computer labs as well as in the hospital, computers are often left on all the time. With 
these assumptions in mind the following compares the Dell Optiplex 755 Energy Star with the Wyse 
terminals used in our evaluation. 

Dell Optiplex 755 Energy Star 

Daily - Formula = High utilization + Low Utilization + Screen Saver + Idle (Monitor off) 









$/Kilowatt 






Hours 


Kilowatts 


hour 


Total 




High Utilization 










6 


0.076 


0.075 


0.034 




Low Utilization 










6 


0.041 


0.075 


0.018 




Screen Saver 










12 


0.046 


0.075 


0.041 


Totals for one business 
day: 


24 




Seven day 


0.094 








week: 


7.000 


Total for work week: 




Weeks in a 


Number of 


0.658 




One week 


year 


PCs 


Total 


Grand total for one 
year: 


$0.66 


52 


500 


$17,117.10 



Wyse VIOL Terminal 

Weekday - Formula = Standard Utilization 





Hours Kilowatts 


$/Kilowatt 

hour 


Total 




Standard Utilization 







Total for one business 
day: 


24 


0.013 


0.075 


0.023 








Seven day 
week: 


7.000 


Total for work week: 




Weeks in a 


Number of 


0.164 




One week 


year 


PCs 


Total 


Total for one year 
terminals: 


$0.16 


52 


500 


$4,258.80 



24/7/365 Blade Center - Formula = Standard Utilization 

Standard Utilization 



24 4.266 0.075 7.6788 

Days in a year: 

Total for one year 
servers: 



365.000 



$2,802.76 



Grand total for one 
year: 



$7,061.56 
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Abstract: Because of its reasonable service price and ease of use, text- 
messaging is popular in Indonesia. Users do not have to own high-end features 
cellphones and lots of pocket money to be able to use text-message services. The 
use of text-messaging affects the way people communicate with one another. 
When text-message communication, which is casual-informal, intersects with 
formal communication between college professors and students, many different 
perspectives can affect the communication process. This paper aims to provide 
perspectives from college professors and students at the University of 
Tanjungpura about the appropriateness of using text-messages for college 
communication. The original paper was unpublished and written in Bahasa 
Indonesia (Indonesian Language). The challenge for rewriting this paper is to 
make sure that translation of text-message quotations from Indonesia into 
English can be ascertained by readers with the same meaning as in the original 
language. 



Introduction 

The popularity of text-messaging in Indonesia continues to grow in recent years. Availability of Twitter, 
Instant Messaging (IM) and Facebook (FB) on cellphone has not replaced the popularity of standard text messaging. 
Compared with Twitter, IM, and FB, standard text messaging is much easier to use. It does not require technical 
skill and it does not have a complicated navigation system. Texting only needs users to be familiar with basic 
features and keypad of their cellphone. Because of its ease in use, accessibility, and low prices, text messaging is not 
easily replaced by new cellphone technologies (Sanders, 2010). 

Owning a cellphone for people in Asia, including Indonesia, is more common than owning a laptop or a 
computer (Cheng, 2010). The reason why cellphones have become a common personal device for most of the people 
in Asia is due to its lower price. Accompanied with the cellphone, users have to purchase communication services 
from the Cellular Telecommunication Services Providers (CTSP). Sending-receiving text messages, sending- 
receiving multimedia messages, having outgoing and incoming calls, and accessing Internet are services provided by 
the CTSP. Text-messaging is a service that has recently increased vastly in almost all Asian countries (Katz & 
Aakhus, 2002; Ling & Baron, 2007; Lipscomb, Totten, Cook, & Lesch, 2007). 

In Indonesia, the growth of the number of text-messaging users can be tracked from CTSP providers, such 
as Telkomsel, Indosat, and proXL. From the year 2008 to 2009, Telkomsel, the pioneer of CTSP in Indonesia, 
recorded a 21 percent increase in use of text-messaging traffic (Telkomsel, 2009). Indosat as one of Telkomsel’s 
competitors recorded an increase of 50 percent text-message users during the year 2009 (Software Pulsa Elektrik 
Elektra, 2009). In the first quarter of 2010, pro-XL recorded a 55 percent increase in the company’s text messaging 
service (ProXL Press Release, 2010). The growth of the number of text message users is a stunning standpoint for 
CTSP opportunity to gain profits. 

For the community, cellphones and other mobile devices have created an additional identity for the person 
who uses these devices. Various concepts were used to describe this identity. Levinson (2004) called this identity 
“mobile hearth”. Watkins (2009) named this identity “net generation” and “digital native”. In an almost similar idea, 
Staid (2008) used the term “mobile identity”. In this paper, we choose to use the term "mobile identity". Mobile 
identity, as in other human identities known well in psychology, has positive and negative elements. The positive 
elements of mobile identity are real-time interaction, better technology literacies, knowledge-able, connectivity- 
collaboration, responsibility, and respect others. In contrast, the negative elements are impatience, information 
overload, over use of intelligence to commit a crime, difficulty in differentiating between public and personal 
business, an increase irresponsible behavior, and no respect to others. 
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Mobile identity is processed into social interaction situation. In the social interaction situation, mobile 
identity is valued by the society based on acceptable social rules. Despite some commonality, every society has 
unique social rules. Acceptable social rules in one society may be unacceptable in another society. To be more 
specific, the way a person does the text messaging may not be similar from one society to another. That is what 
seems to be appropriate in one society may be the opposite one for the other an appropriate way on text messaging 
in one society maybe inappropriate in another society. Therefore, forming a convention on text messaging will 
hopefully make the difference clearly understandable. 

Regarding appropriateness, two values of text messaging are worth discussing. They are first, the elements 
of language use and secondly the safety of people who use text messaging. When analyzing perceptions about text 
messaging communication behavior, as a part of mobile identity, we discovered that there are possibilities of 
disagreement about what appropriate action is and what inappropriate action is. A society can value more to 
language use of text messaging and another society may value more to the safety of text messaging (The & 
Surmiyati, 2010). Forming a convention as a guideline for members to act in the right manner is important. 

Many important aspects of text messaging communication revealed in prior researches, but these researches 
left out attention to the text messaging convention. Kasesniemi (2003) started to reveal the adolescent social reasons 
in using cellphone. Turlow and Mckay (2003) gave attention to linguistic and communication standard practice in 
text messaging. Bell (2006) proposed the fact that young people have reasons to using text message instead of 
calling. Lipscomb et al. (2007) pointed the importance of paying attention to the ethic of text messaging in a 
different place and time. Ling & Baron (2007) focused on the use of text messaging and instant messaging in the 
United States. Tsoukanelis (2008) concerned on the usability and safety of using cellphone for texting. White and 
Kilker (2008) captured the social uses and gratification of text messaging. 

This paper tries to move forward on looking at the convention of text messaging communication, especially 
in college. Studying the convention of text messaging is challenging. First, researches on text messaging will help 
for filling limited resources in this area. Second, relativity of values will influence society on developing the 
convention. Third, translating examples of text message quotation from the original language which is Bahasa 
Indonesia (Indonesian Language) into English will be possible to cause the change in meaning. In observance of the 
challenges, we strive to admit the limitation of this research. 

In college, code of conduct can be used as a reference to assemble text message communication 
convention. In most of the United State code of conduct, convention about text messaging has been covered on code 
of conduct item about communication. In contrast, colleges in Indonesia just started to set in order about this 
convention. Based on this research, we would like to form a foundation for establishing a college code of conduct 
that in the future can be a model for another college or university to implement the same initiative. 

Purpose and research questions 

The aim of this paper is to provide perspectives from college professors and students at the University of 
Tanjungpura about the appropriateness of using text messages for college communication. The research questions 
discussed in this paper are: 

1 . What college activities can and cannot be communicated with text messaging? 

2. How do the professors interpret text messages from students? 

3. How do the students interpret the preference of their professors on receiving the text messages? 

4. What actions should the college apply for having a convention on text messaging communication? 

Method 

For rewriting this paper, we used data collected in 2005. Text messages from students addressed to the 
University of Tanjungpura professors are the major data of this research. Authors and three colleagues initiated to 
collect samples of text messages written by students. The samples chosen were those categorized as “inappropriate” 
or have caused a problem of understanding. Those collections then motivated us to write this paper. Further 
expectation is that the results of discussion will bring new idea of proposing a recommendation to improve the code 
of conduct in the University of Tanjungpura. 

Seven students were voluntarily involved as respondents. Their involvement was for the purpose of raising 
a discussion of what their opinion are about text messaging. Each student was interviewed in an informal situation 
once for approximately 15-20 minutes. Following the interview, authors and students had a group for discussion. 
The involvement of these students was to pursue a balance view of perspective between parties, the professors and 
the students. 
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For the first time, a section of the results from our paper was translated into English and brought to the Ed- 
Media Conference 2010. Four major themes used to describe the most concern of college professor in this paper. 
From this conference, we saw that our participants were curious about what the text messages look like. We 
excluded text message quotations in the first version translated paper because we struggled with worries of 
misinterpreting the original text messages. 

After having some consultation sessions with writing center tutors at the University of Flawai'i, finally we 
have a solution to disclose examples data in this paper. Original text message examples and students opinion will be 
presented in italic. Translation will be followed the original data. In the translation, we adjusted the sentence 
structure to make it enable to understand. There are contingencies that the meaning of the original text will change 
by the sentence structure adjustment. Flowever, we tried the best adjustment to avoid this contingency happened. 

Results 



Text messaging at college 

Using text-message for college persistence is known in Indonesia as "akademik sms" (academic sms). It 
functions to fulfill several purposes, like registration, tuition payment, and announcement. However, in the 
University of Tanjungpura, academic sms has not been implemented. In our data collection discussion, we talked 
over several functionalities that likely to be implemented in our campus. From the discussion, we reached agreement 
and also have some disagreements. In addition, we identified detention factors for implementing academic-sms. 

We reached agreement that registration and tuition payment, college announcement, and grade checking 
services will have benefit to campus if these services are implemented. The following are the benefits that we 
consider to be obtained: 

1. Students will be able to save time for a better activity than to spend their time queuing in front of bank counter 
and registration center. 

2. Students can do the registration and keep getting the update from campus while they are away on vacation. 

3. Campus announcement boards can be used to post pertinent information flyers, such as academic competition, 
scholarship, and jobs vacancies rather than just for announcing the grades. 

4. Increasing students’ confidentiality by keeping the grade private only for personal information when it is sent 
directly to the owner cellphone. 

While as, we could not come to an agreement when the issue is related to using text message to advertise 
courses and send out paper/thesis defense invitation. Advertising courses with a text message is considered doing an 
extraordinary useless job for the professors. The reason lies behind this is that even without text message 
advertisement services, students will automatically take the course, because majority of the courses are required. 
Therefore, we more agree if text message is used to announce important campus events, such as music and dance 
festival schedules, technology and education exhibition advertisement, or seminar and public talk sessions. In term 
of classes’ purposes, text-message announcement services are better used for sending out information about class 
cancellation, class rescheduling, and friendly reminder from professors to students about assignment due. 

Sending out invitation for committee members to attend paper or thesis defense was also an issue that we 
cannot coincided. Paper or thesis defense invitation sent to committee members is a formal document. This 
document is supposed to be sent in a printed version. The invitation should be sent with the copy of paper/thesis in 
advance, at least a week before the defense. Moreover, if the university keeps itself with the current system by 
requiring students to submit printed version paper or thesis for defense, invitation should be sent out using printed 
copy as well. But, there is also a possibility to change the system, such as to combine printed invitation with text 
message or email. Later, if the campus decides to implement information and communication technology widely, 
electronic invitation by email and text message can be adapted. 

After identifying academic sms functions that can and cannot be implemented in the University of 
Tanjungpura, we also exchanged notions on detention factors for implementing academic sms. Three essential 
detention factors for the implementation are: 

1 . Process to introduce, make the technology adaptable and user friendly for faculty and students, and reached the 
level of integration will need some times. 

2. The efficiency and benefit of academic sms should be analyzed in comparison with Internet services that also 
raise its popularity on providing academic services. 

3. The delay in processing and sending out information on time can happen because of some undisciplined faculty 
delay on finishing their grading. 
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The professor interpretation 

Senders and receivers will probably have a different interpretation about context of text messages. In this 
section, we focus on interpretation made by professors of students’ text messages they received. We grouped the 
discussion into five topics associated with the use of greeting, abbreviation, small and capital letters, emotional 
symbol, and clear sentences. 

Although texting is casual and informal, it does not mean that it is allowed to abandon ethics and dignity in 
written communication. In our opinion, highly related with Indonesian culture which is value more on a respect of 
younger people to an elder, students must show their respect to their professors by opening and closing the messages 
with greeting. In practice, students rarely put greeting on the text message they send to their professors. They are 
more straightforward in writing their requests or purposes. For examples: 

Bpk di Kantor kah sekarang 

Mr., are you in your office right now? 

Ini (nama) kpn sy bisa ketemu dgn bpk? 

This is (name), when I can see you sir? 

Saya ijin sakit hari ini 

I [want] a permit [to not attend in class because I am] sick today 

The examples above amongst the professors will be considered less polite. Those Professors will appreciate 
more to text-message that is started and ended with salutations. Having no salutation will give an impression that the 
students have no respect to their professors. That is why we think that texting a greeting is not difficult. Students can 
use simple and appropriate salutation like: 

Pagi/Siang/Malam Bpk/Bu 

Morning/Afternoon/Evening Sir/Mam 
Ass. (Abbreviation for Assalamualaikum) 

Peace be with you, an Islamic way on greeting people 
Trims (abbreviation for terima kasih) 

Thank you 

Wass. (Abbreviation for wassalam) 

Regards, an Islamic way on ending a meeting. 

Using abbreviation for a text message is common. This common does not mean that student could overuse 
the abbreviation. Overusing the abbreviation will make the text message become unclear. For example (Underline 
used to indicate the abbreviation): 

sy tak th di mana t_4_ ktr Bpk skrg? Apa di MM or di FE ? 

I do not know where is your office, Sir? Is there in MM or in FE? 

In this text message the abbreviation of "sy" could be interpreted as "saya" (I), people named "sy", or abbreviation 
for "sekretaris yayasan" (Foundation Secretary). "t4" in this message sender idea is "tempat" (place). The message 
sender used the number 4 which is in Indonesian pronounce as "empat" with the letter "t" to abbreviate the way 
writing the word "tempat". 

This overuse will bother the professor as they have to spend more time understanding the intention or the 
message of the sender. Even more, sometimes the professor has to consult the text-message with their colleague. 
Another example for overuse of abbreviation in student text-message is: 

Klu bsj kamek ke rmh bu t[ ambil skrg soalnya Pig -name- nak brgkt ke Stg besok lusa. Trims & 
maaf sdh merptkan bu. 

If [allowed], we would like to go to your house mam for taking [it] now because Mr. - 
name - will be departing the day after tomorrow. Thank you and sorry have bothered you 
mam. 

In this text message, the abbreviation "bs" can be interpreted as "bisa" (can [I]) or "besok" (tomorrow). "Stg" in this 
message is also unclear. What the message receiver can guess is it may be related with the name of the place. 

We also found that, in writing their text messages, students, mostly, did not pay attention to punctuation 
and the transitional from small to capital letter. For example: 

saya ke kmps now bpk di situ gaks'bab saya man ttd krs yg hrs diserah ke acd hr ini jg 

I will go to campus right now are you there sir because I want you to sign my courses 
registration from which has to be submitted to the academic section today. 
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Sometimes, students write the entire text message in capital letter and without careful attention to 
punctuation. Writing in capital letter for faculty members is impolite manner, because they feel like being shouted or 
being given an order by the message senders. For example: 

BOLEH KAH SA YA KUMPUL TGS HARI INI SBAB SA YA ABSEN KMR 

CAN I SUBMIT MY ASSIGNMENT TODAY BECAUSE I WAS ABSENT 
YESTERDAY? 

Using emotional symbol is acceptable for sending a personal message to friends, but will not be acceptable 
for semi-formal and formal communication in college. The use of emotional symbol on text message will result in a 
different impression for receivers. Therefore, in this group discussion, we all agree that students are not supposed to 
include emotional symbol on their text message addressed to the faculty. In fact, faculties receive lots of text 
message contained emotional symbol such as (Underline used to indicate the emotional symbol): 

Pagi ini sudah saya trima surat recom-nya Bu. Trims bykya. Ibu memang dosen pig baik :-D 

This morning I got the recommendation letter Miss. Thank you. Miss is the best lecturer 
[Laugh] 

B aru saja pulang dr Jkt Pak(-: Bisa saya ke ktr utk konsl. mt kul. besko Pak %-) 

Just got back from Jakarta [smile] can I go to your office to have consultation tomorrow 
sir [confused] 

Besides the faculty perspectives, opinions from students are considered important to be included in this 
study. In the following section, the data was synthesized from limited participants (7 voluntary participants) used to 
give a foundation about what the student may think about their text messages. 

The student interpretation 

Before we start to discuss the major themes revealed from the interview, we would like to present students 
opinion about the relationship between professor and student in the college where the research take place. By 
knowing opinion from students about their relationship with their professor in campus will give a sense about the 
difference between students and professor perspective on text messaging communication. 

The professors want students to respect them as an elder. This respect is represented by students’ action and 
behavior. For example, students concrete action is by giving a greeting to the professor when they come across each 
other, knocking at the professor’s office door to get permission to get into the professor office, are not 
straightforward in showing encounter/disagreement with the professor lecturer, following the rule that is made by 
the professor. Inharmonious personal relationship between student and professor can affect to the accomplishment of 
the course. In other words, students will often fail the course if they do not pay attention to their action and behavior. 
Students’ action and behavior are counted as affective domain of the campus education system, because in 
Indonesia, the education aspects consist of “pendidikan dan pengajaran” (teaching and education). Teaching relates 
with delivery knowledge and skills, while education relates with guiding students to compromise with society norms 
and values. Students’ opinions have illuminated the student-professor relationship that can affect their perception of 
the convention of text messaging. 

According to our respondents students seldom send text message to their professor especially if they do not 
really feel that the message is important. However, even though they feel that their message is important and deserve 
to get quick responds, they do not hope too much that their text message will be responded. Experiencing non 
responded text-message is not something unusual. The following examples show students perception about why 
their messages were not responded: 

Kamek tak tahu pastilah nape tak dibalas...banyak kemungkinan, mungkin gak Bapak tub lagi 
sibuk, atau lagi ke luar kota, atau gak emang tak penting kali sms kamek... hahaha 

We are unsure why [the text-message] do not get reply... [There are] many possibilities, 
probably the professor was busy, or he is out of town, or [probably] our text-message is 
unimportant. [Laughing] 

Sebagai mahasiswa yah kita musti paham, kadang dosen tuh sibuk, punya keluarga, kerja, riset, 
seminar, dan macam-macam lah kesibukan. Jadi belum tentu sms kita itu dibalas. 

As a student, we are supposed to understand [that] professors are sometimes busy, 
[because] they have family [to take care of], work, research, seminar, and other activities. 
So, [that is maybe the reason] our text message are not responded. 

Besides non responded text message, considering the appropriate time to send the message is also 
important. Some professors demand-this and mention it clearly during the class session. For example: 
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Ibu (nama), dosen (matakuliah) saya mengingat supaya mahasiswa tidak menelpon atau sms jika 
sudah lewat jam 10 malam. Sebab, HP ibu tub sudah dimatikan jika dia pergi tidur. 

Mr. (name), [who teach] (course name), reminded students not to call or texting her after 
10 pm. Because, she will turn off the cellphone if she goes to bed. 

Saya rasa sms antara pagi ...mungkin jam 8 udah boleh kali, sampe sore lah. Jam kator atau jam 
kerjanya dosen tub kan tidak ada masalah. 

1 feel [we are able to] send text-message [to our professors] from morning ...about 8 [am], 
probably will be allowed, until after noon. [We are allowed to do so] because that was 
their office hour. Thus there will be no problem. 

The next theme that is frequently mentioned by students is about asking permission to miss the class. Text 
message is a convenient tool to let professor know if the students are unfortunately, have to miss the class. 

However, most of the professors do not like this. For examples: 

Dosen saya mengingatkan agar tidak menggunakan sms untukminta ijin. Jika sakit, maka surat 
keterangan dokter dibawa pada pertemuan berikutnya. Untuk ijin tugas, surat keterangan dari 
kantor bisa di fax ke kantomya dan nanti dibawa lagi satu copy ketika masuk 

My professor reminded students not to use text-message to get permission [not attending 
the class]. If students are sick, medical notice from doctor must be brought to class in the 
next class [session]. For student [who have to] commit on official duty, official letter 
from their office can be facsimile to the professor office and extra copy must be brought 
to class if they attend the class later. 

Teman saya tetap dianggap tidak masuk walaupun sudah minta ijin lewat sms. Kata dosennya, 
tidak salt ijin itu pake sms. 

My friend was considered absent, although he has requested a permission to miss the 
class by sending text message. This was so because the professor stated that [he did not 
counted] text-message as a legitimate way to have a permission. 

Last theme that we consider important to be included in this section is about the felicitousness of students 
to ask their professor preference about text-messaging communication. To ask about professor preference is 
important when professor did not clearly inform the rule on their syllabus. Students tend to agree to ask the 
professor about this regulation. For examples: 

Tidak salah untuk menanyakannya jika dosen tidak menyebutkannya di dalam syllabus. Bcihkan 
itu sangat penting ketika dosen itu tidak pake syllabus. Karena sulit untuk menebak keinginan 
dosen. Biasanya kalau dosennya killer, kita cuma tanya-tanya ke senior tentang maunya Bapak 
atau Ibu itu. 

It is not wrong to ask the professor if they did not mention the rule of text- messaging on 
their syllabus. Moreover, [asking professor preference] is important if they do not [hand 
on] syllabus. Because what the professor want is hard to guess. Generally, if the professor 
is “killer”, we will ask senior about what that Mr. or Mrs. want. 

Maunya kita sib bisa bertanya langsung, tetapi lebih sering kita tanya sama yang sudah pernah 
ngambil matakuliah itu aja. Kalau matakuliahnya wajibyah apa boleh buat, ikuti saja maunya 
dosen itu. yang penting kan kita lulus. 

We want to be able to ask [it] directly, but more often we ask senior student who have 
ever taken the course. If the course is a required one, we will just follow what the 
professor want us to do as long as we pass the course [will be more important]. 

After looking at students and professor opinion, in the following section we will discuss recommendation 
for the college to develop a convention to regulate the text messaging communication. 

Having a convention 

What seems to be a salient consideration is for a college action to regulate text messaging communication 
in in the college’s code of conduct. The regulation then may be taken as a reference for those departments and 
classes to decide whether the type of text messaging is in accordance with the regulation stated in the code of 
conduct. Further action is to incorporate the convention within the departments’ guideline. More specifically, this 
convention can be inserted within the syllabus guidelines by the professors or teachers. If for example, a 
disagreement of text messaging convention exists, the solution may be referred to the established guideline of 
convention. 
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Some colleges prefer to have a convention by proclaiming a ban of not to use cellphone and text messaging 
for communication between professors and students. In our opinion, banning the use of cellphone and text 
messaging for college communication is not a convention. A convention should be a solution that brings 
convenience in communication for both professors and students. Therefore, banning the use of cellphone and text 
messaging does surely not solve this problem. If the banning is enforced, the within campus use of cellphone will 
probably solved but there will not be any guarantee for the similar case of out of campus. As an educational 
institution, college or university is supposed to develop a convention that regulate about good manners on 
communication, including text messaging. 

The University of Tanjungpura has two alternatives for having a convention of communication in the 
university code of conduct. First, rector, as the leader, calls college professors and researchers to conduct a research. 
The research collects data from professors, staff, students, alumni, and society opinion about suitable 
communication convention for college. Based on research results, the university has a senate meeting for revising 
the old code of conduct. Second, initiative action can starts from professors or students. Professors or students, or 
together, have a meeting for discussion. In this meeting, participants discuss and make a document of problems, 
concerns, and solutions about communication convention. The results from discussion are brought by the group to 
the rector and the university senate for follow up. 

Although currently, in a college level, there is no exact convention about text messaging, professor and 
student could start to have it for their classes. Inside the syllabus, the formal written mention of text messaging 
convention can be inserted. Wisely, the professor may repeat it again during class sessions. This is good to inform 
those students who may not be able to attend the class at the very first session and those students that move from 
another section or another class. They will be aware of what they have to be committed to in order to have a mutual 
convenience of communication. If the professor did not mention their preferences on their syllabus, students could 
ask the professor. Usually, when students start to ask about it, the professor will say what they prefer or open a 
forum for discussion on an agreement for the class. 



Conclusion 

Communication with text message in college is unavoidable situation in the digital era. Proclaiming a ban 
of not to use cell-phone and text messaging will not solve the problem regarding the issue of inconvenience or 
convenience feeling for communication between professors and students. Several factors have been identified as a 
cause of the inconvenience for communication between professors and students in the University of Tanjungpura. 
First, different perceptions between the student as the text message sender with the professor as the text message 
receivers about the content of text message. Second, there happen to be preferences amongst professors in receiving 
text message. Lastly, the college did not have a convention about communication stated on its code of conduct. 
Therefore, we highly recommend the college to consider having a convention on this matter. Before the college 
have the convention, professor and student may start having an agreement on what preferences of text message 
model they would accept to be incorporated in their class. 

A salient notice about this research is to cordially state our limitation. First, the number of respondents is 
small that makes it less possible to be generalized as true. Secondly, there may be a different interpretation of 
meaning toward the messages discussed in this study due to the translation problem done by two different sources; 
Indonesian and English. Thirdly, this study was conducted as a pilot study which may not reveal the real essence of 
the problem discussed. 
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Introduction 

The growing popularity of social media and Web 2.0 technologies in online education has led to expectations of 
greater interactivity and self-directed learning. However those expectations have not necessarily meshed with 
reality. Part of this gap has been due to not fully appreciating the difference between the personal or recreational 
contexts in which Millenial students generally use social media (e.g., personal) and formal learning environments 
(e.g., classroom) where participation is linked to a consequential, academic metric. We are interested in what can be 
learned from analyzing how everyday, informal online social worlds create networks and spaces for participation. 
Within the context of a popular social networking site, we present an investigation that focuses on patterns related to 
two questions: (1) How can we characterize patterns of self-directed knowledge sharing on the site? (2) How do 
experts facilitate knowledge sharing and self-direction within the site? 



The Context 

The CarTalk (cartalk.com) discussion board forums came about as a result of the CarTalk radio program, which has 
been a very successful call-in show since the 1970s. The program is made available through National Public Radio 
and has been widely recognized for not only helping non-experts with their automotive problems, but also doing so 
in an entertaining way. Its wide success has spawned a vibrant Web-based community that features active discussion 
boards where fans gather to discuss automotive problems and issues. Perhaps not surprisingly, the CarTalk 
community also offers many other ways to participate and engage fellow community members (e.g., discuss past 
shows, tackle the weekly “Puzzler” question, participate in contests, and contribute to a CarTalk photo album). 

These characteristics establish CarTalk as a Community of Practice (Wenger, McDermott & Snyder 2002). 



Theoretical Perspectives 

In this paper, we briefly define levels of membership and then consider three general theoretical frames: Legitimate 
Peripheral Participation (e.g., Lave & Wenger), arguing-to-learn (e.g., Adriessen, et al. 2003; Jonassen & Kim 
2010), and guided participation (e.g., Rogoff 1998). 
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Defining Levels of Membership 



Within the CarTalk community, we generally construct a focus around two levels of membership: (1) Initiator and 
(2) Expert. An Initiator is the participant who poses a question to the community at-large regarding a specific 
automotive problem or issue for which they seek assistance. The Expert corresponds to any participant who offers 
feedback designed to assist them in this endeavor. 



Legitimate Peripheral Participation 

Towards developing an understanding of the dynamic between Initiator and Expert in the CarTalk community. Lave 
and Wenger’s concept of Legitimate Peripheral Participation (LPP) appears particularly relevant because of the way 
in which it “provides a way to speak about the relations between newcomers and old-timers.” LPP also considers 
“the process by which newcomers become part of a community of practice” (p. 29). In CarTalk, since one of the 
central purposes for the community is to help non-expert vehicle owners troubleshoot problems, many of the threads 
are initiated by “newcomers.” 

As they enter into this network, the “old-timers” typically find the need for more details and so engage in a 
conversation with the newcomer by requesting more details regarding the nature of their problem, and from there 
begins an interactive exchange that builds the thread. This process is inherently a social one in which learning how 
to “talk within a community of practice” assumes as much importance, if not more, than an understanding of 
discrete tools or facts. Lave and Wenger describe this as the difference between “Talking about a practice” versus 
“Talking within a practice” (p. 109). In other words, meaningful participation involves more than knowledge of the 
subject matter content, but also knowledge of the discursive context. LPP is an analytic tool that directs our focus 
towards these social practices and interactions and their role in building a participant’s unique identity within that 
community. 

Central to Lave and Wenger’s concept of LPP is peripherality. On its face, the word “peripheral” suggests notions of 
being on the margins, and when one thinks of periphery as it relates to a community, the common perception is that 
those members who exist in those margins or peripheries represent lesser positions of power or influence in that 
community structure. However, Lave and Wenger envision peripheral participation in a different way. They explain 
that “. . . it’s not meant to suggest a disconnectedness from the ‘practice of interest’” (p. 37), nor is it intended to 
reduce social contexts to a simple binary where the center and periphery are seen as diametric opposites. 



Argumentation 

The second frame of our analysis incorporates argumentation. Common understandings of the word “argument” 
generally envision it as two ideological adversaries going up against one another where one will be ultimately 
determined the winner. In this light, argumentation does not seem conducive to learning. Andriessen clarifies this 
further in relating to people’s perception of argumentation vis-a-vis the popular media. 

Many people think that arguing interferes with learning. They link argumentation to a certain type of oppositional 
argument that is increasingly prevalent in our media culture (p. 443). 

In this context, Andriessen explains that “the goal is not to work together toward a common position, but simply to 
score points” (p. 443). To serve the needs of learning, he introduces a distinctly collaborative form of argumentation 
in which participants are “arguing to learn.” This model promotes four types of benefits: 

1. Elaboration, reasoning, and reflection 
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2. Knowledge of argumentative structures 

3. Development of social awareness and general collaborative ability 

4. Knowledge of group participation norms given that many groups abide by implicit or explicit rules of 

argumentation (pp. 443-44) 



In online contexts, Jonassen and Kim (2010) point to discussion boards as one of the most common venues for 
collaborative argumentation and problem-solving strategies. One specific type of collaborative argumentation is 
“presumptive” which centers on a . goal-directed and interactive dialogue” where “participants reason together to 
advance arguments by proving or disproving presumptions” (p. 444). This same sort of goal-directed, presumptive 
argumentation appears in CarTalk where different Experts argue the details of diagnoses and proposed solutions. 



Guided Participation 

Our third theoretical frame relies on Guided Participation. Rogoff asserts that Guided Participation is not a model or 
technique but centers on 

“. . . the processes and systems of involvement between people as they communicate and coordinate efforts 
while participating in culturally valued activity” (Rogoff 1995, p. 142). 

Rogoff further clarifies Guided Participation by seeing it as three, interacting planes: personal, interpersonal, and 
community/organizational. Rather than hierarchical levels, Rogoff stresses that these should be seen as interacting 
planes because while they may vary in degrees of intensity, they nevertheless inform one another in significant ways 
(Rogoff 1998, p. 688). In a context that bears considerable similarity to our focus, Heo and Breuleux (2009) use the 
Interpersonal Plane to investigate interactions among participants in an informal community of online food service 
professionals. 



Methodology 

We consider this to be an initial, exploratory case study whose orientation is primarily qualitative by relying on 
Grounded Theory (Glaser & Strauss) and where the CarTalk discussion board network serves as our unit of study 
(Yin, 1993; Stake, 2000). Our quantitative component includes basic frequency counts. Our exploratory sample 
consists of approximately 52 threads that occurred between approximately March 31-April 6, 2010. We did not 
restrict our sample to any specific categories of discussion and so it encompassed a broad range of topics related to 
vehicle maintenance and repair. Ultimately, our analysis generated seven interpretive categories. 



Code 


Description 


Story-Sharing 


Expert provides knowledge to an Initiator by way of a 
personal or professional learning experience 


Vetting of Ideas 


Expert attempts diagnosis of Initiator’s problem by 
presenting idea or theory for examination by 
community members; it often involves argumentation 
among experts 


Precedent 


Expert references and elaborates on previously stated 
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opinion, analysis, or proposed solution 


Explicit Instruction 


Expert provides explicit series or sequenced set of 
instructions related to troubleshooting or resolving 
problem; it also includes instructions presented as “if- 
then” statements 


Tacit Instruction 


Expert provides background information or general 
knowledge related to the problem, but not specific 
steps for troubleshooting it 


Facilitating Question 


Expert asks Initiator for further details that are 
considered crucial to diagnosing and troubleshooting 
problem 


Resource Sharing 


Expert includes reference to external link or source 



Findings 

Among these categories, the three which show the highest frequency are Explicit Instruction, Precedent, and Vetting 
of Ideas. It is perhaps quite understandable that Explicit Instruction would show the highest frequency since the 
nature of the CarTalk community is to help people solve mechanical problems that typically require careful attention 
to the sequence with which a solution is carried out. Precedent appears as more of a surprising result because of the 
potential for suggesting levels of community cohesion and sustainability. In this context of our analysis. Precedent 
reveals two significant details: (1) Experts validating the efforts of others in the community, and (2) Experts using 
argumentation to learn the reasons for the legitimacy of a specific position (e.g., solution to a problem). The Vetting 
of Ideas involves collaborative experimentation. This celebrates the prototypical mechanic who loves to tinker under 
the hood. Unlike the lone, tinkering mechanic, however, the CarTalk community plays a vital role as a feedback 
mechanism. 





Code 


Frequency 


1 


Story- Sharing 


56 


2 


Vetting of Ideas 


129 


3 


Precedent 


162 


4 


Explicit Instruction 


220 


5 


Tacit Instruction 


118 


6 


Facilitating Question 


92 


7 


Resource Sharing 


27 
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Code 1 : Story Sharing 



In one example of Story Sharing, the Initiator shares a story about a neighbor who has had difficulties getting a local 
car dealership to honor a warranty since the original owner was taken over by another company. The Initiator further 
explains that the new owner will only honor repairs under warranty if the neighbor can present a copy of the repair 
receipt. The Initiator believes this claim is specious, but is not absolutely certain and so turns to the CarTalk 
community for feedback. 

The specific instance of Story-Sharing occurs about two-thirds of the way into the discussion thread when an Expert 
relays his own story about confusion over a warranty. Specifically, he explains a major engine problem with his first 
car, but since the dealership had closed by the time this happened, he thought that the warranty would be useless. 
Many years later, while in the process of re-organizing some old files, he came across the old paperwork from this 
“first car” experience. Upon closer examination, he realized that what he had originally believed to be the warranty 
was actually an insurance policy that was handled by a company completely separate from the dealership. He 
continues to explain that where originally he was inclined to blame the dealership, this subsequent discovery showed 
him that this was not the case. Notable here is the Expert’s use of a story to establish a sense of shared connection 
with the Initiator and to convey the value of considering an alternative viewpoint. 



Code 2: Vetting of Ideas 

We saw participants engage in Vetting of Ideas when Experts attempted to diagnose an Initiator’s problem by 
presenting an idea or theory for examination by community members. In many of the discussion threads, we also 
observed Experts engage in strategies of “Arguing to Learn” (Andriessen, 2006; Jonassen & Kim 2010) as they 
debated the merits of competing solutions. 

One excerpt we analyzed comes from a rather extensive discussion with a deceptively simple beginning. The 
discussion begins with the Initiator relaying an incident in which he assisted a couple whose car battery had died. In 
relaying the details of what he did, he explains that it was an older car and required more than jumper cables. To 
work around this obstacle, he did what computer aficionados would call a “hack” (e.g., hackaday.com) in which he 
manipulated a coil wire near the engine in a particular way that successfully restarted the engine. He explains that he 
learned this hack from his father, but confessed that he had no idea why it worked. Logically, he chose to title the 
thread, “Why does it work?” 

The question sparks an engaging debate about what underlying electronics and physics might explain the hack. At a 
relatively early point in the thread, one Expert interjects with a fairly strong dissenting opinion by asserting that the 
solution does not involve a complex interaction of electronics but rather a simple release of excess gas pressure that 
happened when the Initiator manipulated the wire. Although his language suggests a strong belief in his position, he 
concludes by noting his openness to competing viewpoints — a key characteristic of Vetting. So with Vetting we 
often see this two-pronged nature in which argumentation and exploration (of ideas or theories) go hand-in-hand. 



Code 3 : Precedent 

Somewhat related to Vetting of Ideas were instances of Precedent when Experts would publicly recognize the 
contribution made by a previous participant. In most situations, the recognition would surface in the form of a 
compliment regarding the accuracy of a specific assertion or the appropriateness of a recommended solution to a 
problem. Periodically, it might also assume the form of an elaboration or a disagreement. 



326 




In one example, the Initiator relays a problem of a sticky gas pedal and efforts to fix it by cleaning the butterfly 
valve (flow regulator); however he is tentative about implementing this corrective action and specifically worries 
that it might result in more harm than good. In the ensuing discussion thread, some Experts tentatively legitimize the 
Initiator’s proposed solution, but acknowledge potential dangers to the throttle plate. At the end of the thread, we 
observe one Expert use part of his first sentence to explicitly recognize the accuracy of a previous Expert’s 
recommendation regarding this potential danger (i.e., the throttle plate). This scenario illustrates how an Expert 
draws the Initiator’s attention to the resources of the community already in existence (i.e., Precedent) and implies a 
conscious awareness of the community as a knowledge base. 



Code 4: Explicit Instruction 

Explicit Instruction emerges as one of our broader categories as it can assume the form of sequenced instructions or 
“if-then” statements. In one discussion thread we analyzed, the Initiator requests help for a car that has difficulty 
starting when he turns the ignition. He further explains that sometimes he has to rev the engine up to around 2000 
RPM to prevent it from turning off, but when he does this, a strong scent of gasoline and plumes of “white colored 
exhaust” emerge. In developing a solution, one Expert expresses confusion about the specific nature of the ignition 
problem (e.g., “Does it crank and crank and crank and just won't fire up?”) and accordingly, wants to probe for more 
details in order to generate a more accurate diagnosis. 

Several noteworthy features emerge in this instance of Explicit Instruction. First, the Expert organizes his response 
in relation to the Initiator’s concerns by using a conditional structure. In the first paragraph, he specifies the 
condition of not smelling the gasoline and begins the second paragraph the converse - not smelling the gasoline. He 
constructs these condition-based instructions in the form of “if-then” statements (e.g., “If you don’t smell gasoline, 

. . . [then] turn the key to start . . . [conversely] If you smell gasoline . . . [then] pop the hood . . .”). Also noteworthy is 
how subsequent instructions are expressed in the form of short imperatives (e.g., “turn the key”, “pop the hood”, 
“look at both ends”). These are commonly recognized characteristics of well-written instructions (Barker 2003). 
Finally, the Expert’s response also indicates a desire to extend the length of this conversational exchange by 
requesting an update from the Initiator after he implements these instructions. This suggests how requests for 
updates sustain the overall life of the thread. 



Code 5: Tacit Instruction 

We recognize Tacit Instruction emerging in threads where an Expert provides background information or general 
knowledge related to the problem, but does not go so far as to present specific steps for troubleshooting it. 

In one discussion thread we analyzed, the Initiator requests help on a problem with lug nuts on her van. She conveys 
how she discovered that they had not been properly tightened after a recent trip to a service center. After having 
them fixed, she expresses concern that her van may have sustained some other type of damage during the time 
between the initial problem and subsequent repair. As the Experts consider her problem, their conversation generally 
gravitates towards a discussion of best practices regarding if and when lug nuts should be retightened after 
installation. About halfway through the discussion, one Expert suggests that they should be retightened after 100 
miles. In response to this point, the Initiator expresses being unaware of this “best practice.” In the next thread, 
another Expert clarifies the earlier point by explaining that this 100-mile rule applies only to vehicles with aluminum 
alloy wheels. 
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Notable in this segment of the thread is how the Expert who offers the clarifying information is not responding to a 
direct question posed by the Initiator nor explicit instructions; rather it is background information that explains the 
context to which the 100-mile rule applies. 



Code 6: Facilitating Question 

An instance of a Facilitating Question occurs when an Expert asks an Initiator for further details that are considered 
crucial to diagnosing and troubleshooting problem (e.g., type of car, weather conditions, duration of problem). 

In one discussion thread that represented a fairly typical pattern, the Initiator is thinking about buying a used car and 
plans to take it to a mechanic to have it looked at for potential problems. Before he does this, though, he wants to 
have a specific checklist of items for the mechanic to investigate and asks the community for help in doing this. 
Before any Experts in the community begin to recommend criteria for the checklist, the first reply appears as a 
Facilitating Question that asks for fundamental details about the vehicle (e.g., make, model, year, and mileage). 

Facilitating Questions arguably represent one of the clearest indicators for distinguishing Initiator from Expert. If 
not immediately, then fairly early in the progression of the discussion, we observe an Expert ask for further details 
regarding the vehicle in question. By directing Initiators to first identifying core details of the vehicle, they 
emphasize the importance of diagnosis. 



Code 7: Resource Sharing 

As the term suggests. Resource Sharing occurs when an Expert includes an external hyperlink or source. In CarTalk, 
where participation is completely voluntary, we consider this noteworthy because Experts are taking additional 
initiative to find a resource that falls within the general parameters of the Initiator’s request. 

In one discussion thread of this type, the Initiator describes a problem related to jump-starting a car. The specific 
problem he describes is that he is unable to properly attach the clamp to the positive side of the battery terminal 
because it is in an extremely narrow space. In one of the responses, an Expert suggests the possibility of a jump- 
starting it with a remote device. Interesting here is how he arrived at the suggestion of a remote terminal since the 
Initiator did not specify the vehicle’s make and model, and the implication is that a comparatively small percentage 
of cars accommodate this option. One possible explanation is the identifying tags that the Initiator originally put 
into his post. Tags are keywords that function as additional metadata for electronic communication and are 
commonly offered as options in discussion board forums. Typically, tags are used to facilitate retrieval, but in this 
case, the Expert may have used them to deduce further contextual details. By seeing the tags “Porsche” and “911”, 
he may have deduced that remote terminals are an option that Porsche 911 vehicles can accommodate. Indeed, later 
in the discussion the Initiator expresses appreciation for the remote terminal proposal and a likely intent to purchase 
one. 



Conclusions & Implications 

As an exploratory study, our conclusions are necessarily tentative. With this in mind, though, we see two patterns 
that show potential for enhancing our understanding of knowledge sharing and self-directed learning in online 
environments. In CarTalk, we frequently see Experts use argumentation as a means to share and facilitate 
knowledge; however, a critical component to this is the Initiator whose question sets that activity in motion. On the 
second front, Story-Sharing emerges as a potential lens for gaining greater insights into self-directed learning. Some 
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Experts opt to embed directive advice within broader personal narratives that further contextualize solutions. 
Additional context may enrich an Initiator’s understanding and instill a desire to learn more. 



The argumentation that occurs among the Experts reveals an interesting connection to peripheral participation (Lave 
& Wenger). In the course of responding to an Initiator, the Experts, as described earlier, frequently forward different 
arguments as to what they believe is the best solution to the Initiator’s problem. In some cases, these positions 
exhibit considerably different approaches, while in others, they differ by small degrees. What is notable in this then 
is that learning appears to happen as much as among the Experts as it does among the Initiators. In this light, the 
Initiators play important roles in further expanding the learning of the Experts since it is their initial question that 
served as the opportunity for that learning to take place. While the Initiators in CarTalk may be viewed by the 
Experts as peripheral participants, especially those posting a question for the first time, they nevertheless play a 
valuable role in the development of that community of practice. But to what degree are the Experts aware of this 
peripheral impact? Would it change the nature of their interactions with the Initiators? These questions merit further 
investigation. 



Elements of self-directed learning appear to surface in story-sharing. On first thought, it would seem more logical 
that Initiators would be more likely to engage in storytelling since they would presumably lack the same level of 
technical vocabulary as Experts and would compensate by explaining it through stories. Yet we observed Experts 
regularly resort to stories as a way of connecting to Initiators. They would intersperse their answers with anecdotes 
from their own personal experiences as a professional mechanic or a self-taught “Do-It-Yourselfer” (DIY’er). As we 
saw in the Story-Sharing example, the Expert used a personal story as a way to reach out and establish a sense of 
shared experience. Only after first making this attempt at a personal connection, does the Expert move towards the 
more practical goal of offering more directive feedback. The stories that are shared and told by DIY’ers represent an 
intriguing constituent of the CarTalk community as they bring an intrinsic passion to the topic. Indeed, a simple web 
search can reveal a vast DIY culture that has spawned a whole range of communities that offer support and informal 
instruction on a wide range of topics (e.g., Instmctables.com, Lifehacker.com). DIY culture has also attracted the 
attention of academic researchers. For example the lnstmctables website represents an offshoot of an MIT Media 
Lab project, and more recently, the Living Environments Lab at Carnegie Mellon has studied more than 2000 people 
involved with DIY projects (e.g., Dorkbot, Etsy). While DIY’ers certainly possess an intrinsic passion for their 
topic, what is more interesting is their enthusiasm for sharing it. What factors encourage this enthusiasm for sharing? 
What audience does the DIY’er have in mind when sharing? How does this shape the style and substance of the 
artifact that is shared? 



Overall, our initial case study encourages more attention to the impact of peripheral participation and story-sharing. 
Peripheral participation suggests moving away from binaries of high activity and low activity cliques and more 
attention towards the impact of certain discursive acts (e.g., questions). Story-Sharing may suggest the benefits of 
extending reflection to a more public space. It may suggest that knowledge-building is not just about aggregating 
facts but accommodating affective elements that allow participants to argue about what they mean. 
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Abstract 

This proposal describes the technology mentoring experience of a graduate student and faculty member at a 
mid-western university in the U.S. The purpose of this case study was to understand how student technology 
mentors can facilitate faculty members’ movement through the Innovation-Decision Process (Rogers, 2003) when 
considering adoption of social networking tools into teaching. Findings suggest that technology mentors play a 
critical role in helping faculty members overcome obstacles during the Innovation-Decision Process and serve as 
imperative communication channels during the process. 

Descriptors: technology mentoring, social networking, innovation-decision process 



Purpose and Importance of the Study 

Web 2.0 offers opportunities for information- sharing, collaboration, and social networking that are 
unrivaled in our history (Gale, 2008). As these tools become increasingly present, opportunities for their 
implementation into teaching and learning become more innovative and engaging. Flowever, such tools are often not 
utilized in classrooms due to the small number of teacher-education programs with faculty who are modeling 
instructional methods that integrate the use of such tools into their teaching (Carlson & Gooden, 1999; OTA, 1995). 
This lack of modeling is exacerbated by the fact that schools are spending only a small percentage of technology 
dollars on professional development (Ansell & Park, 2003) despite the fact that teachers say they need more of it 
(Flutchison & Reinking, 2010). The current study describes how one mid-western university implemented a 
technology mentoring program for faculty to encourage the adoption of Web 2.0 social networking tools into their 
instruction. This alternative form of professional development paired graduate students in a technology mentoring 
course with faculty members who were considering adopting social networking technology innovations into their 
instruction. 

This purpose of this study was to explore how student technology mentors can facilitate faculty members’ 
movement through the Innovation-Decision Process (Rogers, 2003) when considering adoption of Web 2.0 
innovations. The Innovation-Decision Process comes from Rogers’s (2003) Diffusion of Innovations Theory which 
describes a model for studying the adoption of technology in higher education and educational environments. His 
theory posits that those considering technology adoption move through five stages: (1) knowledge, (2) persuasion, 
(3) decision, (4) implementation, and (5) confirmation. When proper communication and support occur in each 
stage, there are more opportunities for adopters to fully adopt the new technology. 

The goal of the mentoring process was for faculty members to increase their implementation of technology 
into teacher education courses. Increasing technology integration, particularly with new tools, is important because 
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faculty members in teacher education programs serve as important models for pre-service teachers on how to 
integrate technology into teaching. The hope of the mentors was that they would be able to support faculty 
members’ movement through the Innovation Decision Process (Rogers, 2003) in a way that made technology 
adoption both easier and more likely. This study examines the experience of one mentor-mentee pair. 

Methods and Data Sources 

This study utilizes a single case study approach (Yin, 2009) to gain an understanding of how a student 
technology mentor and faculty mentee experience the exploration of technology innovations through the lens of the 
Innovation-Decision process (Rogers, 2003). The following research question guides this study: How can a student 
technology mentor support movement through the Innovation-Decision Process (Rogers, 2003) for a faculty member 
who is considering adopting Social Networking tools into her instruction? The participants were a doctoral student 
in a Curriculum and Instructional Technology program and a faculty member in a department of Curriculum and 
instruction at a mid-western university. The doctoral student served as a technology mentor for the faculty member 
as part of a technology mentoring course. Students in this course are paired with a faculty member and each pair 
spends one hour every week working on technology related developments. 

This program provides faculty member an individualized approach to professional development because 
each faculty member can choose the focus area based on his/her interests (Zachariades & Roberts, 1995). The 
current study reports the experience of one mentor-mentee pair. The data sources for this study include personal 
reflections by the student posted to a blog and feedback from peers in the technology-mentoring course. Reflections 
were posted weekly and included information such as personal feelings about the mentoring experiences, frustration 
faced during the meetings, specific procedures or steps for each module or program, and observations about the 
meetings with the faculty member. Data was analyzed using constant comparative methods (Strauss & Corbin, 

1998). Postings were initially analyzed to identify themes and look for evidence of movement through the 
Innovation-Decision Process. Codes were developed for emerging themes and evidence. Codes and themes were 
further refined until no new ideas emerged from the data. 

Results 

Results from this study demonstrate how a student technology mentor can enhance and facilitate a faculty 
member’s progress through the Innovation-Decision process (Rogers, 2003) when considering if and how to 
implement a technology innovation into their teaching. Results from the current study indicate that student 
technology mentors can serve vital roles in supporting movement through the process, particularly when a faculty 
member faces obstacles in any part of the process. An additional theme that emerged from the data was 
communication. 

How Technology Mentors Support the Innovation-Decision Process 

This section provides evidence for how the student technology mentor in this case study supported the faculty 
member’s movement through the Innovation-Decision Process. It is divided into sections that describe each stage. 

Support During the Knowledge Stage 

In the current case, the student technology mentor helped the faculty mentee gain knowledge about the 
advantages of disadvantages of using social networking tools as a replacement for traditional course management 
systems throughout the knowledge stage. The technology mentor played an important role in this stage because she 
introduced the faculty member to Ning, a social networking tool that the faculty member was familiar with but not 
as a replacement for a course management tool. The faculty member had previously decided that Ning would not 
suit her needs because she had not viewed it as a course management system. The technology mentor helped the 
faculty member overcome this obstacle by providing her with knowledge about how Ning could be used to suit her 
needs, despite the faculty member’s previous notions about it. The following quote from the student’s reflection 
notes demonstrates how the technology mentor supported the faculty member in the knowledge stage. 

10/13/09. The faculty member and I started to explore Ning. Later on, the faculty mentee surprisingly found 
out that she has an account in Ning already. And then I showed her how to use some features in Ning. She 
is an active learner, and she always wanted to experience the steps on her own. That’s very’ positive and 
kept me wanting to show her more things than was on the planned agenda. 

During the meetings, the faculty’ mentee gained a lot of knowledge in using Ning’ s features. Although she 
did not have enough time to construct everything required to use Ning in her classes, she tried her best to 
test every’ feature right away and absorb the knowledge using hands-on experiences. 
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Support During the Persuasion and Decision Stages 

In the persuasion and decision stages the student technology mentor played a critical role as a 
communication channel for the faculty mentee as she negotiated her feelings toward the new technology innovations 
introduced by the mentor. Individuals typically form negative or positive attitudes about an innovation during the 
persuasion stage. In this case, the technology mentor facilitated a positive attitude toward the technology innovation 
by helping the faculty member overcome the obstacles that initially caused the faculty member to view social 
networking tools negatively. Further the mentor supported the faculty member through the decision stage by acting 
as a communication channel as the faculty member struggled with her feelings about the disadvantages of the social 
networking tool under consideration. This excerpt from the mentor student’s reflection illustrates the faculty 
members’ movement through the persuasion and decision stages. 

In late October 2009, 1 started to share with my faculty mentee how to use Ning as a course management 
tool and provided some teaching ideas. When the faculty mentee saw the layout of Ning, she was attracted 
by the fact that it includes the blog feature. So, students can create their own blog entries and are able to 
see classmates ’ blogs as long as they are in the same network. Compared with Moodle, this feature is more 
flexible than Moodle ’s features. Before that meeting, I was confident that the faculty mentee would use 
Moodle for her next semester ’s teaching, but she chose Ning instead of Moodle. This is the characteristic of 
the decision stage. In this stage, individuals prefer to try’ the innovation in their own situation and then 
reach the adoption decision. The faculty mentee had to think carefully about her course and consider the 
content of the course before she made her decision. 

Support During the Implementation Stage 

In this stage, the student mentor promoted the integration of the social networking site, Ning, into the faculty 
member’s courses by supporting the faculty member’s use of the innovation in her own context and diffusing 
uncertainties about the new technology. The mentor again served a critical role in helping the faculty member 
overcome obstacles that were not anticipated. 

Support During the Confirmation Stage 

Finally, during the confirmation stage, the student mentor confirmed the faculty member’s decision to 
implement the technology innovation by providing technological and affective support to the mentee. Again the 
mentor served as an important communication channel for the faculty member as she looked for confirmation that 
her decision was appropriate each time she struggled through an obstacle. The following excerpt is from the 
mentoring student’s reflection at the end of the semester. 

The importance of interpersonal communication channels and support continues at the confirmation stage 
of the innovation-decision process. Individuals look for supportive messages that confirm their decisions. 
Because using Ning as a CMS was still new to the Dr. Smith, she would ask me for confirmation and kept 
asking questions. Whenever she expressed her concerns and we figured out ways to address them, Dr. 

Smith (pseudonym) would seem more confident about her decision. At the end, The investigated programs 
during with the semester were Web 2.0 applications, Moodle, Ning, Google Map, Second Life, Wimba Tool, 
WebCT and Mac software. Dr. Smith made her decision about using Ning for her children ’s literature 
course. However, she needed to wait until her students were enrolled in her class and then she could start 
to make arrangement on how she would run her course. 

Communication 

An unanticipated theme emerged from the data in addition to the evidence of how the technology mentor 
supported the Innovation-Decision Process. The idea of good communication was prominent in the student 
reflections and class discussions and played an important role while implementing the new innovation. As 
mentioned in the five stages of Innovation-Decision process, the mentor always served as an important 
communication channel. In the whole process, the mentor established several communication methods which 
facilitated faculty member’s innovation. Weekly meetings with face-to-face demonstration provided the faculty 
member chances to ask questions. E-mails and Facebook discussion also allowed the mentoring pair to interact with 
each other after each meeting. Moreover, the communication was not restricted to only between the mentoring pair, 
but the whole community in the same department through the class Facebook page that was established for the 
group. Blog entries with comments from peers about the mentoring experiences served as an important 
communication tool for the technology mentor. Because communication was encouraged and facilitated through 
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multiple modes of digital response, new ideas about technology integration also emerged. This kind of environment 
also changed the dynamics of learning and mentoring. Everyone become the agent of change in the process of 
technology integration. 



Discussion 

The main purpose of the Innovation-Decision process is to motivate individuals to reduce their uncertainty 
about an innovation and its outcomes. Although providing customized mentoring to faculty for technology 
integration through student mentors is a unique approach to professional development, the findings of this study 
confirmed that the new mentoring program was successful in helping the faculty member integrate technology into 
her classes. The faculty member was motivated to learn more because the goal was well customized and organized. 
Plus, by following the Innovation-Decision process model, the mentoring pair was able to fully experience each 
stage to make sure whether the new tool fulfilled the instructional need. The faculty member also received just-in- 
time support from student mentor. After implementing the tool, the weekly demonstration helped the faculty 
member to understand each feature and allowed her to explore. This approach also helped to decrease the faculty 
member mentee’s uncertainty about instructional technologies. This approach may have worked so well because of 
the social nature of the process, supporting Roger’s (2003) idea that “diffusion is a very social process that involves 
interpersonal communication relationships” (p.19). Both the mentor and mentee were engaged in conversation and 
reflection about the innovation. Without the opportunity for interaction with others about the project, the faculty 
member may have taken a longer time to complete the innovation or may even have withdrawn in the middle of the 
process. The support and communication channel from the community helped the faculty member see the value of 
experimenting with new tools. Furthermore, the ease of use of Web 2.0 also reinforced faculty member’s confidence 
level for implementation. At the end, the mentoring pair was able to create an ideal pilot model which was helpful 
for integrating social networking into literacy course as a course management system. 
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Abstract 

This paper shares the authors’ experiences in developing and implementing an Online Master of Science (M.S.) 
degree program in Instructional Technology (IT), the WebIT program, at The University of Tennessee (UT), during 
2008 2010. The paper focuses on the organization, implementation and funding of the degree program rather than 
details of the curriculum design. The overall attrition rate in the first WebIT cohort (N=25) was 56%; 12 students 
withdrew from the program and 2 students were removed from the program due to academic difficulties. Data and 
interpretations regarding possible links between program organization and attrition are discussed. 
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Implementing an Online M.S. Degree Program in Instructional Technology: 

Promises and Pitfalls 

The purpose of this paper is to share the authors’ experiences in developing and implementing an Online 
Master of Science (M.S.) degree program in Instructional Technology (IT), the WebIT program, at The University 
of Tennessee (UT), during 2008-2010. The focus of the paper will be on bigger picture issues of the organization 
and implementation of such an effort rather than the details of the curriculum design. This paper was written 
following the completion of the first cohort cycle of the program, a two-year period in which a group of students 
who were recruited during the Spring semester of 2008 completed their Masters degree work in Instructional 
Technology, at the end of the Spring semester of 2010. The first students to matriculate in the WebIT program 
constituted the WebIT 1 cohort. 

The College of Education, Health and Human Sciences (CEHHS), the Department of Educational 
Psychology and Counseling (EPC) and the UT Department of Distance Education and Independent Study (DEIS) 
provided startup funding for the WebIT project. Startup funding for the program consisted of approximately $12,000 
and the allocation of existing personnel resources, as follows: (a) three Graduate Teaching Associate (GTA) 
positions, (b) a half-time assignment for one faculty member to coordinate the WebIT development and 
implementation efforts, (c) a summer workshop ($2000), (d) personnel support ($5000), and (e) a server to run the 
Moodle Course Management System ($5000) (Moodle Trust, 1999). 

The online curriculum for WebIT was planned as a variation of the existing M.S. in Instructional 
Technology curriculum at The University of Tennessee. The primary difference between the two versions of the 
curriculum was that the online curriculum would be delivered via a combination of Computer-Mediated 
Communication (CMC) tools with limited face-to-face (f2f) contact between students and instructors, whereas the 
on-campus curriculum is predominantly f2f with students and faculty engaging in limited online interaction to 
augment weekly classroom-based meetings. Existing IT courses were redesigned for delivery using a suite of CMC 
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tools selected by the faculty. The courses taught during WebITl used a combination of Internet e-mail, listservs, 
Moodle (Moodle Trust, 1999), Wimba (Wimba Incorporated, 2009) and specialized websites to constitute the 
WebIT virtual classroom environment. The WebIT program was designed to be collaborative in nature. Each course 
was actively led by an instructor, and each WebITl student was expected to engage in a combination of individual 
and socially-mediated instructional activities such as those described by Chandler, Levin & Levin (2002); Kowch & 
Schwier (1997); Levin, Levin & Chandler (2001); Levin & Waddoups (2000); Angelino, Williams & Natvig (2007) 
and Palloff & Pratt (2007). 

Three advanced graduate students in Instructional Technology were hired as GTAs and served as the 
primary instructors for the WebITl sections of the M.S. curriculum, but the courses were taught in collaboration 
with IT faculty. The IT faculty actively mentored the GTAs (who served as apprentice online instructors) and 
monitored the WebIT courses as part of their ongoing teaching responsibilities. During each semester two GTAs 
were assigned instructional responsibilities while the third was dedicated to providing technical support to students 
and instructors. IT faculty taught on-campus sections of the same M.S. courses at the same time as WebIT online 
courses were taught so faculty and GTAs could collaborate on teaching activities and responsibilities. 

Curriculum Delivery Model 

Cohort admission model. The cohort program design model is often described in the literature (Levin, et 
al., 2001; Levin & Waddoups, 2000) as being beneficial because it promotes close-knit working relationships among 
students in online programs. This characteristic is thought to be important because these relationships constitute a 
kind of learning community that can provide a dynamic and supportive social structure for individual learners 
(Chyung, Winiecki & Fenner, 1998; Obrien, 2002; Rovai & Wighting, 2005; Wegerif, 1998). Individuals without 
such relationships can experience a sense of social isolation that Carr (2000) characterizes as being 
lost-in-cyberspace. Rovai and Downey (2010) contend that “Students in distance learning programs may be more 
likely to experience isolation and alienation from the institution because of their physical separation from the school 
and its services and from other students” (p. 145). 

The WebIT faculty selected a cohort program model for the WebIT program as opposed to an 
open-enrollment program model because the cohort model is group-based and facilitates the development of 
intra-group, supportive, working relationships among the members of the cohort. Such a structure seems to offer 
better opportunities for supporting students’ academic engagement and reducing the sense of personal isolation 
reported by some online students (Chandler, et al., 2002; Chyung, et al., 1998; Palloff & Pratt, 2007). 

Cohort pacing. A very critical aspect of an academic program is how much time and effort are required by 
the student to complete the program. Masters degree programs typically seem to vary between 24 and 36 semester 
credit hours. Thus, completing a typical Masters degree program can take a student between one and three years. UT 
Masters degree-seeking students typically take two years to complete the on-campus M.S. in IT degree program. 
However, there is a fair amount of variance around that average. Many factors influence the pace at which 
on-campus students pursue academic work and the time required to complete the M.S. degree program. 

During the Spring of 2008, just prior to starting WebIT, the IT faculty conducted a Needs Analysis (via 
online survey) to determine the level of interest among potential students for an online M.S. degree program in IT 
offered by UT. The targeted population was K-12 teachers from across the state of Tennessee. A group of 122 
self-selected respondents completed the survey. These respondents represented 25 of the 98 counties in Tennessee 
(25%) and half of them (50%) lived more than 100 miles from Knoxville, while nearly one-third of them (30%) 
resided more than 200 miles from Knoxville. 

Ninety respondents (74%) in this sample indicated that they would prefer to complete an online M.S. 
program in less than two years. Seventy-one respondents (58%) indicated that they would prefer having the option 
of taking longer than two years. When asked if they would be willing to take two courses per semester for two years, 
52 respondents (42%) said that they would be willing to work at that pace to complete an online M.S. program in IT. 
While these data are not perfectly clear, one logical interpretation is that the students would prefer an online 
program that provides options so that they can “set their own pace” (79%). Nearly three-quarters (74%) would 
prefer to complete the program faster, but more than half (58%) would work longer, if necessary. In terms of the 
two-year pacing, it is hard to know if these responses were influenced more by the overall duration of the 
commitment or the six-hour per semester effort obligation. Logically, most students would wish to finish faster, 
whenever possible, but concern about the amount of work that such a pace might require is the likely explanation for 
the lower rate of agreement (42%) regarding taking two courses per term for two years. 

These Needs Analysis data, our experiences with our on-campus student population, and our limitations in 
personnel capacity led to the decision to deliver the WebIT program in a two-year-long cohort format. Students 
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began the program by taking one course in the initial Summer semester. During each of the five subsequent 
semesters of the program, students completed two semester-long courses. 

WebIT curriculum. The WebIT program consists of 11 semester-length (3 credit) courses, 33 graduate 
credit hours. The first iteration of the program, WebIT 1, was delivered over a two-year period, beginning in the 
Summer of 2008. The curriculum for the WebITl cohort is shown in Table 1. 

Table 1 

Curriculum Sequence for the WebITl Cohort 



Term 


Course 

Number 


Course 

Name 


Credit 

Flours 


June, 2008 




On-campus, 3 -day orientation 
workshop 




Summer 
(July, 2008) 


IT 575 


Internet: Implications for 
Teaching and Learning 


3 


Fall, 2008 


IT 521 


Computer Applications 
in Education 


3 




IT 573 


Introduction to Multimedia 


3 


Spring, 2009 


IT 570 


Instructional Systems Design 


3 




IT 576 


Advanced Interactive 
Multimedia for Instruction 


3 


Summer 

(June- July, 2009) 


IT 578 


Instructional Media 
Development 


3 




CS 504 


Teachers, Schools and Society 


3 


Fall, 2009 


IT 566 


Administering Instructional 
Media: Networks 
and CMC tools 


3 




EP 532 


Online Collaborative Learning 


3 


Spring, 2010 


IT 577 


Internet-Mediated 
Collaborative Learning 


3 




EP 582 


Educational Research 
Fundamentals 


3 


Total 






33 
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Distance Education Program Funding Model at UT 

The theory. The funding model for Distance Education (DE) programs at UT is a unique process that is 
separate and distinct from the two other funding models (annual fiscal-year and summer school) that operate on the 
campus. The DE funding model is more entrepreneurial than either of the other two funding models because it is 
based upon a mechanism that directly links the tuition revenue generated by student enrollments to the academic 
department that delivers the instructional program. 

Academic departments at UT that choose to offer DE programs receive compensation for the effort that is 
based upon the number of students served, the number of hours delivered, and a dollar-multiplier directly related to 
the amount of tuition paid for each credit hour by each student. The academic department delivering the program 
does not receive the entire amount of tuition paid by the student, but the academic department does receive a 
significant proportion of the tuition paid by each student. This proportion (approximately 76% of the campus base 
tuition rate was returned to the WebIT program) was stable during the WebITl cohort but has since been decreased 
and is subject to further changes. The funds generated by student tuition payments in an academic year are paid out 
to the academic department at the beginning of the next academic year, i.e., one year in arrears. Each Fall semester, 
the campus pays a lump sum to the academic department that represents the total revenue that was generated by the 
program through the delivery of the DE curriculum during the previous academic year. 

The reality. The WebIT admissions committee admitted 25 students to the WebITl cohort. Prior to the 
summer workshop, two students dropped out of the program, so 23 students attended the required face-to-face (f2f) 
workshop in June, 2008. Prior to summer enrollment in the first WebIT course, two other students dropped out of 
the program. One student dropped out of the summer course. During the Fall of 2008, three students dropped out. 
During Spring 2009, four more students dropped out. In addition, one student was removed from the program for 
poor academic performance. During the Fall of 2009, one additional student was removed from the program due to 
poor academic performance. Eleven students enrolled for each of the two courses offered in the final semester of 
WebITl, Spring semester, 2010. Eleven students completed the WebIT M.S. degree program at the end of Spring 
semester, 2010. 

Table 2 summarizes the student attrition data for the WebITl cohort . Three findings from these data are 
fairly obvious: (a) four students (29% of those who dropped) were lost before the academic portion of the WebIT 
program began, (b) only one student was lost from the program during the second year, and (c) the reasons provided 
by students for dropping the program conform to two broad categories often mentioned in the literature as: personal 
and institutional variables (Kember, 1989; Moore, Bartkovich, Fetzner & Ison, 2002; Rovai & Wighting, 2010; 
Tinto, 1993). 

The attrition data also generally conform to those shared by Chyung, et al. (1998). The students described 
in her study dropped early in the academic program and for reasons similar to those shared by the students who 
dropped out of the WebIT program. 
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Table 2 



WebITl Cohort Student Attrition Data 



Student 

Number 


Term 

Dropped 


Reason 


1 


Pre- Workshop 


None Given 


2 


Pre- Workshop 


None Given 


3 


Pre-Enrollment 


Program Emphasis Mismatch 


4 


Pre-Enrollment 


Inability to secure financial aid 


5 


Summer 08 (1) 


Busy Personal Schedule 


6 


Fall 08 (2) 


Program Emphasis Mismatch 


7 


Fall 08 (2) 


Personal Flealth Problems 


8 


Fall 08 (2) 


Pace of work; Personal Issues 


9 


Spring 09 (3) 


Time Management Problems; 
Personal Issues 


10 


Spring 09 (3) 


Loss of Financial Aid Support; 
Family Responsibilities 


11 


Spring 09 (3) 


Family Responsibilities 


12 


Spring 09 (3) 


Pace of work; Family 
Responsibilities 


13 


Spring 09 (3) 


Admission revoked for poor 
academic performance; 
Pace of work; Family 
Responsibilities 


14 


Fall 09 (5) 


Admission revoked for poor 
academic performance 



Note. Numbers in parenthesis indicate the semester (1-6) during which students were lost from the WebIT 
program. 
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The characteristics of the UT funding model and the attrition associated with the WebITl cohort present 
unique challenges for maintaining the WebIT program. Because the funding formula itself is not stable, cash flow is 
highly constrained and enrollments can be unpredictable, it is nearly impossible to predict the level of funding that 
will be available to support the program in subsequent years. Based on WebITl, a strong first year will likely 
generate sufficient funds to deliver a second year; but a weak second year may not generate sufficient funds to start 
another WebIT cohort. Therefore, the success of the WebIT program, in terms of its self-sufficiency and 
maintainability, is dependent upon two factors that are highly unpredictable from year-to-year. 

Promises Realized During the WebITl Cohort 

What WebIT offers UT. The WebIT program offers great potential for the Instructional Technology 
programs at The University of Tennessee. Based upon our experiences with WebITl, all of this potential was 
realized and could likely be maintained in future. First, WebIT offers the prospect of being able to reach beyond the 
boundaries of the campus to recruit students who otherwise would not be able to matriculate at UT. Second, WebIT 
provides the opportunity to construct a research laboratory for examining many issues related to the successful 
design and implementation of online degree programs and electronically-mediated, collaborative learning 
environments. Third, the WebIT program provides an ideal vehicle for helping our senior doctoral students obtain 
critical, real-world experience as teachers in electronically-mediated instructional environments. Fourth, the WebIT 
program offers the IT faculty an opportunity to acquire additional fiscal resources to support doctoral students and 
faculty in the Instructional Technology programs at UT. 

What WebIT offers the students. Based upon both faculty and student perceptions, the cohort structure 
was highly valued by students and, though slightly more costly to maintain (initial workshop expenses, program 
coordinator and additional, special course sections for cohort members) the cohort model seems justified as a 
programmatic structure that helps address the unique needs of distance learners (Rovai and Downey, 2010; Rovai & 
Wighting, 2005). Without a cohort structure, it is possible that program attrition might have been higher, but this is 
purely conjecture. What we know is that the successful WebITl students reported a strong preference for the cohort 
organization. 

Though attrition was high during the first year of the WebIT program, the students who chose to continue 
in the WebIT program appear to have bonded to a degree that is difficult to compare with students in 
open-enrollment programs. They voluntarily participated in two, f2f, optional professional development activities 
(the annual Tennessee Educational Technology Conference in 2008 and 2009) and demonstrated their knowledge 
and skills by making presentations and assisting others in making presentations. 

It is not clear to what extent the friendships among cohort members begun during WebITl will last beyond 
the completion of the WebIT program, but it was evident to the WebIT faculty and GTAs that the members of the 
cohort group valued one another and enjoyed their involvement as members of the cohort. In response to an Exit 
Survey question that asked each of the WebITl students about the potential likelihood of his/her maintaining any 
relationships with other members of the cohort beyond the completion of the W ebIT program, 1 0 of the 1 1 W ebIT 1 
students (91%) who completed the program, provided an affirmative response. A typical response was the 
following: “Yes. Many of the relationships I have formed will continue through friendships as well as perhaps 
professionally [sv'c]”. 

The members of the WebITl cohort appeared to exhibit a level of comfort with one another that is more 
often seen among people who have worked closely together for many years, which seems remarkable given the 
relatively short duration of their acquaintance during the WebIT program. The cohort group dynamic evident in 
WebITl provided a sharp contrast to the group dynamic typically observed among students in f2f M.S. courses 
populated through open enrollment. 

Pitfalls Encountered During the WebITl Cohort 

Though a number of minor problems were identified and resolved during the implementation of the first 
cycle of the WebIT program, two significant problems were identified that defy simple solutions. The first problem 
is that of student attrition. The second is the nature of the funding model for the WebIT program. These two 
problems are related but each requires a unique solution to improve the viability of the WebIT program. 
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Student attrition. During the WebITl cohort, the WebIT program experienced an attrition rate of 56% 
(40% during the completion of academic coursework). This is similar to attrition rates reported by others in recent 
years (Bowser, 1992; Chyung, et al., 1998; Diaz, 2002; Kember, 1989; Moore, et al., 2002; O’Brien, 2002; Rovai & 
Downey, 2005; Rovai & Wighting, 2010) but obviously not what was desired. One possible interpretation of the 
high rate of attrition is that the WebIT program is simply too rigorous. A second possible interpretation is that the 
pre-matriculation workload expectations of the WebIT students was inconsistent with the reality of what was 
required to be successful in the WebIT program. A third possible interpretation is that the structure of the WebIT 
program was not sufficiently supportive or conducive to the development of a community of learners, i.e., the 
WebIT program did not succeed in promoting student engagement and developing a sense of community among all 
participants. Undoubtedly, there are other possible explanations for why WebIT student attrition was so high, but 
given the experiences of others in this regard, it is highly likely that the high rate of attrition experienced during the 
WebIT program is directly related to some combination of these three critical factors. 

Additionally, several program-specific characteristics also likely influenced student decisions regarding 
persistence. The program sequence for WebITl was a rigid, two-year timeline that required students to participate in 
a specific sequence of courses offered at a fixed pace (2 courses per semester) over a specific period of time (2 
years). For those unable to set such a pace, it may have been easier to find an instructional program that was more 
flexible in this regard. 

Our interpretation of the high attrition rate experienced during WebITl is that the WebIT program was 
challenging and pushed students to adopt a pace of work that is beyond what a significant fraction of them could 
manage, given their significant personal, family and work-related responsibilities. Advertising media in the United 
States are rampant with messages aimed at potential online students who, for whatever reason, cannot disengage 
from their work or personal situation and physically attend an institution of higher learning. These messages seek to 
convince potential students of the convenience of online degree programs. Many online programs are extolled as 
being anytime and anywhere but they fail to accurately portray the many challenges involved for students seeking to 
use new media, new tools and new social conventions (all of which require significant time to learn) in order to 
master a new content; while at the same time remaining fully engaged in their everyday lives. 

DE funding for WebIT at UT. The second significant problem that the WebIT program encountered 
during the first cohort cycle is related to the characteristics of the UT DE funding model. Three main problems 
surfaced for the WebIT program in connection with this funding model. The first problem is that the EPC 
Department was required to fund the entire first year of the WebIT program because no tuition-return funds would 
be available until the beginning of the following academic year. The second problem is that since summer school is 
funded separately at UT, the first summer WebIT course had no source of funding. The third problem is the 
requirement that all tuition-return funds paid to the academic department at the start of the academic year must be 
expended during that academic year. 

Of these problems, the third is the most problematic for running a small-scale entrepreneurial program like 
WebIT. Failure to resolve the third problem, coupled with inconsistent student enrollment, would likely trigger a 
recurrence of all three problems in each successive cohort cycle. The requirement to spend all tuition-return funds 
in the same academic year in which they are generated means that a program like WebIT cannot adequately manage 
funds from year-to-year, to ensure program continuity. The amount of tuition-return generated in AY 08-09 (first 
year) and provided to the EPC Department to run WebIT in AY 09-10 (second year) was approximately double what 
was minimally required to run the WebIT program during the second academic year. However, the loss of students 
during AY 08-09 (first year) meant that the funding generated during AY 09-10 (second year) that would be 
available to start the second WebIT cohort (WebIT2) in AY 10-11, would be insufficient to fund the minimal 
requirements to run the first year of the second WebIT cohort (WebIT2). 

Without a funding model that would permit the WebIT program to bank funds in times when the 
tuition-return exceeds expenditures and use those funds to address program requirements in times of need, the EPC 
Department would continually be placed in the position of having to find funds to jump-start new WebIT cohorts. 
Likewise, the EPC Department would reap the fiscal reward of absorbing all of the excess tuition-return every other 
year. The latter outcome would not be problematic for the EPC Department, but the oscillation between feast and 
famine is problematic and counter-productive to the concept that entrepreneurial DE programs such as WebIT 
should operate on a self-sufficient basis. Further, the campus DE funding model does not allow for the most logical 
and timely application of program resources to the solution of program problems, such as long-term planning. 

Obviously, these two problems, student attrition and finding model, are related. Using the UT DE funding 
model, high attrition during the first year of a two-year cohort can create a funding problem for the first year of the 
subsequent cohort and would require that the academic department continually find ways to fund the first year of 
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subsequent two-year cohorts. The WebIT program could increase enrollments slightly, but larger numbers of online 
students (without additional instructional resources) will likely generate other problems (Angelino, et al., 2007; 
Palloff & Pratt, 2007; Tomei, 2006) and may not sufficiently address the funding shortfall issue. Larger course 
enrollments might generate an environment where less student-teacher interaction and support are possible, thus 
increasing the lost-in-cyberspace effect and resulting in even larger attrition that, in turn, would result in lowered 
tuition revenues. 

At present, the WebIT initiative at UT is under review to determine whether or not it will be possible to 
deliver the program in a manner that is both educationally and fiscally sound. The WebIT 1 students provided a 
wealth of information to inform these deliberations and the members of the IT faculty are analyzing these data to 
determine the feasibility of continuing the WebIT initiative. 
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Abstract 

Interdisciplinary team design has potential to create interactive environment and quality instruction for 
cyberlearning. There are limited researches on how instructional designers collaborate in practice. This qualitative 
study explores the role of instructional designers in collaborative design, how they collaborate, and the desired skills. 
The initial data analysis shows 1) instructional designers play a central role in team design; 2) they use formal and 
informal channels to communicate; and 3) the desired collaboration skills include negotiation, problem-solving, 

time-management, and facilitation. 

Introduction 

Cyberlearning has become the main stream of education (Allen & Seaman, 2008). Educators need to 
understand who their students are, how they learn, and what they need to know in order to support students to be 
successful in this fast-changing world. Rapid advances in technology, neuroscience, and learning science open new 
ways to efficient and effective instructional design practice to provide students’ better cyberlearning experience and 
promote meaningful learning in cyber environments. 

Cyberlearning calls for a more interactive environment, learn centered focus, prompt feedback, and an 
engaging and effective online learning community (Institute for Higher Education Policy, 2000; Kearsley. 2001; 
Palloff & Pratt, 2001). Recent development of instructional design models suggest that interdisciplinary team design 
can be a successful approach for online course design and development. In this collaborative design approach, team 
members share expertise, “learn from one another, broaden their knowledge, and appreciate the strengths which the 
other members bring to the table” (Care and Scanlan, 2001, p. 6). Although many studies have been done regarding 
what instructional designers actually do in an instructional design practice and these studies seem to focus on 
individual instructional designers (Christensen & Osguthorpe, 2004; Pieters & Bergman, 1995; Rowland, 1992; 
Winer & Vazquez-Abad, 1995), little research was found on how to make instructional designer’s collaboration 
more effective in a team design. This qualitative study is designed to investigate how instructional designers 
collaborate with other experts in a team design and what collaboration skills are expected to make such collaboration 
effective. 



Literature Review 

Team design has been suggested by many organizations and researchers as a method to ensure the quality 
of education provided by institutions (SREB, 2001; Care & Scanlan, 2001; Levy, 2003)). For example, the Southern 
Regional Education Board (2001) encourages institutions and states to build an instructional design team for a 
quality online environment. Such a team might consist of the instructional designer, graphic/interface designer, 
technical support personnel, content expert, direct instructor, information resource personnel, mentors/tutors, and 
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assessor. The instructor, however, remains at the center of the team to guarantee academic integrity, with the 
assistance from other partners. Levy (2003) suggested an organizational structure change in online educational 
program. This change should involve different people who do different jobs. Care and Scanlan (2001) have also 
advocated an Interdisciplinary Team Model where various participants meet as a team on a regular basis to develop 
the course, solve problems, and discuss issues as course development unfolded. The participants are content 
specialist, instructional designer, student representative, media specialist, program director, and external faculty 
member. 



In instructional design practice, subject matter experts, instructors, technologists or other personnel may all 
serve in the capacity of experts to inform instructional design. Smith and Ragan (2005) strongly recommend that 
instructional designers build relationships with experts and obtain rich and accurate information from them through 
venues such as interview. Dick, Carey, and Carey (2008) introduced the idea of instructional design in teamwork to 
design, develop, and implement teaching and training. Thus, communication and collaboration skills are critical for 
team functioning throughout the instructional design process and the eventual generation and delivery of products 
and service. 



Methodology 

In order to investigate how the team design approach works in instructional design practice, this case study 
was conducted and guided by the following research questions: 

1 .# What roles do instructional designer and other experts play in a team design? 

2. # How do they collaborate with each other during their design process? 

3. # What collaboration skills should an instructional designer possess? 

Participants 

A total of eleven faculty and staff from a north central university participated in this study, including four 
instructional designers (faculty status), two subject matter experts/developers (faculty status), two librarians, one 
multimedia specialist, one IT manager, and one adjunct faculty (part-time instructor). 

Data Collection and Analysis 

Structured, semi-structured interviews and observations were conducted from March to November 2009. 
Artifacts were collected in the same time frame. Two investigators completed one-hour training on coding the 
interview transcripts. They agreed upon appropriate coding methods during the training and blind reviewed 
interview transcripts individually. The themes that emerged from the data were recorded in Word documents. Then 
they met and discussed the themes from their findings until an agreement to be reached. 

By the time this proposal is completed, the researchers analyzed the transcripts of three interviews. Two are 
from instructional designers and one is from subject matter expert. The following themes were emerged from the 
data. 

Roles and tasks of each member in the team 

In this team approach, instructional designers play a central role, connecting all the team members. The 
instructional designer also serves as the project manager who leads the team and pushes course development 
forward. The developer provides the contents and work together with the instructional designer to create 
assignments, activities, research, and projects. The developer also facilitates the workflow of the adjunct faculty 
members and seeks their inputs through online retreats. The librarian supports the instructional designers to locate 
relevant journals and articles and have them approved by the subject matter expert. The instructional designers send 
out invitation to multimedia specialist and discuss details for creating proper multimedia pieces to enhance students’ 
cyber learning. The adjunct faculty is responsible for delivering the course and providing students’ feedback to the 
developer. The IT manager ensures the technical part of the team design process. 
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Communication channels and collaboration methods 



Communication channels for this team design approach are classified as informal and formal 
communications. Informal communications include stopping by each other’s office, emails, and phone calls. One 
designer creates designated folders in email accounts to save and track the communication occurred among the team 
members. Formal communication channels include a 2 to 6 hour block time for course development only, a Skype 
conference meeting, and Google Docs. 

Collaboration skills for instructional designers 

All the participants agreed on the importance of communication skills for an ID. Other important skills 
needed in team design include negation skills, problem solving skills, facilitating skills, decision-making skills, 
synthesizing skills, persuasive skills, conflicting solving skills, time management skills and leadership skills. 

Conclusions 

Although only the collected data have been partially analyzed by far, the results of the study are promising. 
The interdisciplinary team works best for the cyberlearning design. The ultimate collaboration could be achieved if 
every team member understands each other, develops good collaboration skills, and employs effective 
communication channels. A more in-depth understanding of improving collaborations in a team design will be 
reported once the data is fully analyzed. 
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Abstract 

This papers describes the process of creating Web 2.0 based WebQuests. It also discusses students’ responses to it. 
The Web 2.0 WebQuest has three defining features (1) Web 2.0 design style (2) integration of Web 2 services and 
trends such as blogs, social networking sites and wikis (3) use of Really Simple Syndication (RSS). In one class, my 
students and I created regular WebQuests and Web 2.0 based WebQuests. In another class, my students and I used 
these WebQuests. Data were collected from both classes through interviews and observations. Results indicated that 
while students enjoyed the WebQuest activity, the responses from the Web 2.0 style groups were more positive. 
Introduction 

The Web has brought new possibilities for teaching and learning. WebQuests are among the most used 
methods by K-16 educators as a strategy for integrating the Web into classroom instruction. The WebQuest 
technique was created by Bemie Dodge in 1995 with early input from Tom March. A WebQuest can be defined as 
“an inquiry-oriented activity in which some or all of the information that learners interact with comes from resources 
on the Internet” (Dodge, 1997). This technique usually requires students to work in groups and heavily relies on 
information sources on the Web. Students are guided to visit appropriate web resources to look for information. 

A WebQuest usually includes six steps: (a) the introduction; background information about a topic that 
learners will explore (b) the task; a general description of what learners are required to do (c) information sources; 
mostly web sites for students to investigate to complete the task (d) a description of the process; a step by step 
description for learners to follow (e) evaluation; includes evaluation criteria for students’ works and (f) the 
conclusion; a summary of the experience. WebQuests can be a short term (one to three class periods) or a long term 
(one week to a month) activity and usually require group work. Furthermore, WebQuests can be designed within a 
single discipline or be interdisciplinary (Dodge, 1997). 
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The WebQuest method has been widely adopted in K-16 education (Zheng et al, 2008). One important 
point to mention is that while being applied to a wide variety of different educational settings, the WebQuest 
technique has changed little (Abbit & Ophus, 2008). Since the creation of WebQuests, however, the Web has 
changed a lot, shifting from Web 1.0 to Web 2.0. Web 1.0 was the first and Web 2.0 is the next phase of the Internet. 
Web 1.0 and Web 2.0 differ on how they present information. Although Web 1.0 was mostly about retrieving 
information, Web 2.0 represents a more collaborative, interactive, and user- focused approach. 

Coombs (2007) states that Web 2.0 is ‘transforming the Web into a space that allows anyone to create and 
share information online-a space for collaboration, conversation, and interaction; a space that is highly dynamic, 
flexible, and adaptable’ (p. 17). Web users are now easily building pages, sharing photos and videos, and interacting 
with each other. These users are generating most of the content by uploading videos, pictures and other media, 
participating in discussions (for instance leaving comments on others’ writings, videos, pictures), and writing 
(blogging, wikis etc). This is why many call Web 2.0 a more user-powered and democratic approach than Web 1.0. 

With Web 2.0, a new design style was also introduced. This style is a more simplistic approach. In this 
style, special attention is given to usability; making web site designs easy to browse for Internet users. This includes 
presenting information in a clear way and not causing confusion for the users. Presenting content is very 
straightforward. Users do not spend time trying to find what they are looking for from complicated web site layouts. 
Usually, in this layout, there is content centered in 1 or 2 columns, large texts, plenty of white space to direct 
attention to the content, fewer graphics but more small icons and solid areas of different content sections. 

Research studies indicate that a large portion of the young population (under 20 years old) use Web 2.0 
sites such as social networking (e.g. facebook.com, myspace.com etc.) or video/photo sharing sites (e.g. 
youtube.com, flickr.com etc.) (Balci & Ayhan, 2007; BECTA, 2008; Pew Internet, 2007). Kurt (2009, p. 36) argued 
that: 

Most of the WebQuests, if not all, could be found to carry very basic characteristics of Web 1.0 in terms of 
functionality and design. These WebQuests do not interact with the learners. One of the major weaknesses 
of the WebQuests is that the design structure of this method is not interactive with learners. Interactivity 
here means the extent to which learners can actively participate in the process and collaborate with other 
learners. That means this technique heavily relies on written content. It does not fully take advantage of 
current Web technologies which may increase interactivity and learners’ engagement dramatically. 
WebQuests should embrace the benefits of Web 2.0 in order to engage today’s learners with the 
information we introduce through this method. 
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Thus if we created a WebQuest based on Web 2.0, would that be more effective than using a traditional WebQuest 
as a teaching method? 

Methods 

To create Web 2.0 WebQuests, we employed Blog and Wiki software and services to create our Web 2.0 Web 
Quests. Blogs and Wikis were used to accomplish many things such as increasing collaborations and discussions 
among students, adding many dynamic features such as polls, video and picture galleries, discussion boards etc. and 
monitoring students’ activities by employing history features. Another advantage of using blogs and wikis as a basis 
for our WebQuests was that they simplified the design aspects of this technique by making use of templates, which 
are pre-developed web site page layouts. Furthermore, we used Really Simple Syndication (RSS) to increase the 
effectiveness of the WebQuests we created. Via RSS, we syndicated content from other such sites as Youtube.com. 
RSS also enabled us to create automatically updating WebQuests. 

I teach two educational technology courses and one of the important topics in these courses is the 
WebQuest technique. One class is called Computer Literacy and the other one is called Educational Technology and 
Material Design. In computer literacy, I teach WebQuests as a technique. In the other class, my students and I try to 
create different educational technology materials from PowerPoint to web sites. We also create WebQuests. In this 
class, we tried to create Web 2.0 based WebQuests. We tried to create a more dynamic and user-focused WebQuest 
with the help of the Web 2.0 tools and services described in this paper. Our Web 2.0 WebQuest has three distinctive 
features different from the traditional WebQuests: (a) Web 2.0 WebQuests carry the characteristics of Web 2.0 
design styles (Scratchmedia Limited, 2009; Chang, 2006) (b) Web 2.0 WebQuests include many such dynamic 
features as searching, commenting, discussion boards etc. (c) Using RSS to syndicate content from related Web 
sites. 

Creation of Web 2.0 WebQuests 

I randomly divided my students into five groups. Each group consisted of 7-8 students. Each group was 
required to create two versions of a short term WebQuest; (a) a Web 2.0 WebQuest (b) and a traditional style 
WebQuest. For the Web 2.0 WebQuest, each group used WordPress blogging software to create their WebQuest. 
WordPress is the most popular blogging system today (W3techs, 2010). One distinctive feature of WordPress is that 
WordPress can run multiple blogs from one installation of WordPress. This feature let us easily create multiple 
blogs for each group. Each group was also required to create the same WebQuest using the original WebQuest 
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templates found at http://webquest.org/index-create.php . Since I taught these students Computer literacy previously, 
they knew how to design simple Web sites. 

I also established a separate Wiki page for students’ discussions for each WebQuest. Each student was 
required to write what he/she was doing and respond or comment about others’ Wiki entries. More specifically, 

Wiki was used to help students to manage and organize information, share resources and discuss their work during 
the creation of the WebQuest activities. I closely followed the Wiki pages to monitor students’ activities. Wikis have 
a history feature, this means that whenever there is a change, Wiki saves the before and after version of the change. 
This was a useful feature to follow students’ progress. 

WordPress has numerous themes; pre-designed web site layouts. A majority of them are designed by 
professionals and their licenses are suitable for educational use. These themes can be found at 
/wordpress. org/extend/themes/. This means that students were easily able to change the look of their WebQuests. 
Furthermore, the features of WordPress can drastically be increased by using their plugins found at 
http://wordpress.org/extend/plugins/ . Students were free to use whatever themes or plugins they like to achieve 
what they want in their WebQuests. 

Data Collection 

Data were collected from my Computer Literacy class students and Educational Technology and Material 
Design class students. As I stated above, the WebQuests were created in my Educational Technology and Material 
Design class. Then, I used these new Web 2.0 WebQuests and the original style WebQuests to teach the WebQuest 
technique in my other class, Computer Literacy. Data collection included unstructured observations and interviews. I 
conducted unstructured interviews with 10 students (5 students from the Computer Literacy group and 5 students 
from the Educational Technology and Material Design group). These students were randomly selected. All the 
interviews took place in a private room. Each interview lasted around 15 minutes, and a tape recorder was used to 
record the interviews. Thereafter, the data was transcribed and analyzed. Each interview was independently 
examined. This examination included coding then, determining subcategories, and finding emerging themes and 
issues. In general, I wanted to learn about the students overall perceptions about the WebQuest technique and 
experiences using or creating WebQuests. 

Results 
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In general, the interviews suggested that although students’ from both classes enjoyed the WebQuest activities, 

students’ responses to the Web 2.0 style WebQuest were more positive. Here are some responses: 

Student from Computer Literacy Class: I liked both versions of the WebQuests, but I think the Web 
2.0 WebQuests offered more interactivity. I enjoyed that we were able to do many things on Web 2.0 
WebQuests like adding our comments and participating in polls. 



Student from Educational Technology and Material Design class: During the creation of these 
WebQuests, Web 2.0 style WebQuests offered more flexibility. We could add any functionality we 
wanted. The difficult part was that we created this Web 2.0 WebQuest, and then were required to 
create the same WebQuest using basic HTML [referring to traditional WebQuest]. That was 
impossible to do. We only did it partly because Web 2.0 WebQuests have so many features that we 
could not accomplish in our traditional style WebQuest. 



Student from Educational Technology and Material Design class: We were able to finish our Web 2.0 
WebQuests much faster than the traditional WebQuests. Our Web 2.0 WebQuests look more 
professional than our traditional WebQuests. 



Many of them stated that they liked how it looked and they liked the features available more than the other, 
traditional, WebQuest. I also observed that students using this new WebQuest were more focused, created more 
discussions, and produced more groups’ thoughts in the WebQuest activities than the traditional WebQuest. 
Furthermore, in the Web 2.0 WebQuest’ s Wiki pages, there were a lot more activity than in the traditional style 
WebQuest Wiki pages. 

Discussion and Conclusion 

This paper describes how my students and I created Web 2.0 based WebQuests. There are numerous 
studies examining the impact of WebQuests on students’ learning (for a review of studies, see Abbit & Ophus, 
2008). Yet, there is little knowledge of the factors affecting a WebQuest’ s success or failure. However, it seems that 
Web 2.0 style may have an influence on a WebQuest’ s effectiveness. There are many reasons that Web 2.0 styles 
WebQuests are potentially more effective. More comprehensive studies for all grade levels are required in order to 
conclude that Web 2.0 WebQuests are more effective. 

WebQuest is a Web based technique. This means that special attention should be given to Web 
accessibility. Current WebQuests are known to have substandard design elements (Kurt, 2009). This means that they 
lack many accessibility standards. From this experience, we learned Web 2.0 WebQuests were more accessible than 
traditional WebQuests because students employed WordPress themes. These themes are designed by professional 
web designers and thus are consistent with accessibility standards. 
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One important point to mention is the importance of security when designing Web 2.0 WebQuests. As I 
stated we used WordPress software to build our Web 2.0 WebQuests. Since WordPress is a very popular tool, 
millions of Web sites are using this software. This popularity means that individuals who want to make trouble try 
to find ways to crack into accounts or sites to cause damage. To prevent any damage, keeping your WordPress 
installation, including the plugins up-to-date is essential. 
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